Convolutional attention-based adaptive separation network for eeg artifact removal

Abstract

The application discloses a kind of based on convolution attention for electroencephalogram artifact removal adaptive separation network, including sequentially connected encoding module, separation module and decoding module;Encoding module contains an input module, three sub-modules A and three down-sampling modules, three sub-modules A are sequentially connected, and a down-sampling module is connected after each sub-module A;The separation module includes the sub-module B combined by multiple time series convolution modules, PRelu activation function, point convolution, sigmod activation function;Decoding module includes three sub-modules C, three up-sampling modules and output module, three sub-modules C are sequentially connected, an up-sampling module is connected before each sub-module C, and the output of third sub-module C is connected with output module.The application can efficiently extract clean electroencephalogram signal from contaminated raw electroencephalogram, and it is important to automatically and real-time pre-processing of electroencephalogram signal.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of data processing, and particularly relates to a convolution attention-based adaptive separation network for electroencephalogram artifact removal. BACKGROUND

[0002] Electroencephalogram (EEG) is an electrical signal recorded by sensors on the scalp to record the electrical physiological activity of the cerebral cortex, and is a main method for non-invasive acquisition of brain signals. It has great significance in the fields of cognitive psychology, brain-computer interface (BCI) and diagnosis of nervous system diseases. However, EEG signals are weak and have high temporal resolution, and are easily disturbed by noise generated by experimental equipment and artifacts caused by physiological activities, thereby producing various artifacts. The sources of these artifacts include measuring instruments and human activities. The artifacts generated by measuring instruments can be avoided by precise recording systems and strict recording procedures. However, the generation of physiological artifacts is complex and difficult to eliminate, and it includes eye movement artifacts (EOG), electromyographic artifacts (EMG), electrocardiographic artifacts (ECG) and external artifacts (motion artifacts, vascular wave artifacts). Studies have shown that physiological artifacts can seriously affect the acquisition of real characteristics of electroencephalogram signals and interfere with the subsequent analysis of EEG signals.

[0003] The biggest challenge in electroencephalogram artifact removal is to identify and remove artifacts while preserving as much potential neural activity information as possible. Traditional artifact removal methods mainly include regression, filtering, blind source separation (BSS), wavelet transform (WT), empirical mode decomposition and hybrid methods. However, most of these methods rely on prior analysis experience and are difficult to automatically remove electroencephalogram artifacts. These methods are only suitable for specific types of artifact removal, and the arrangement and selection of decomposed electroencephalogram components and artifact components have not been effectively solved. Moreover, traditional methods have high computational complexity and are not suitable for online applications. Electroencephalogram artifact removal methods need better time and precision efficiency, and the use of deep learning methods to achieve effective automatic artifact removal is a current research hotspot.

[0004] With the rapid development of deep learning, it has been widely used in image denoising, speech denoising and other tasks in recent years. At the same time, inspired by these tasks, it also has a certain effect in the removal of EEG artifacts. Gao et al. (Gao T, Chen D, Tang Y, et al. EEG Reconstruction With a Dual-Scale CNN-LSTM Model for Deep Artifact Removal[J]. IEEE Journal of Biomedical and Health Informatics, 2022, 27(3): 1283-1294.) proposed a dual-scale model combining convolution and LSTM, which is beneficial to the extraction of the spatiotemporal features of EEG signals. Enhance the denoising effect of the model. Sawangjai et al. (Sawangjai P, Trakulruangroj M, Boonnag C, et al. EEGANet: Removal of ocular artifacts from the EEG signal using generative adversarial networks[J]. IEEE Journal of Biomedical and Health Informatics, 2021, 26(10): 4913-4924) proposed a gan model to remove ocular artifacts, which can adaptively generate pure EEG signals from noisy signals. Yi et al. (Yi P, Chen K, Ma Z, et al. EEGDnet: fusing non-local and local self-similarity for 1-D EEG signal denoising with 2-D transformer[J]. arXiv preprint arXiv:2109.04235, 2021) designed a denoising model based on the transformer architecture, which reduces the complexity of the model by slicing the EEG signal as input. Chen et al. (Chen J, Pi D, Jiang X, et al. Denosieformer: A Transformer based Approach for Single-Channel EEG Artifact Removal[J]. IEEE Transactions on Instrumentation and Measurement, 2023) designed a variational auto-encoding model combining U-Net and Transformer, which has better denoising effect than the advanced methods in recent years.Zhang et al. (Zhang Z, Yu X, Rong X, et al. A Novel Multimodule Neural Network for EEG Denoising [J]. IEEE Access, 2022, 10: 49528-49541) designed a multi-module denoising model, which passed the output of the previous module to the next module as input, achieving good denoising effect. Yu et al. (Yu J, Li C, Lou K, et al. Embedding decomposition for artifacts removal in EEG signals [J]. Journal of Neural Engineering, 2022, 19(2): 026052) proposed a method combining encoding and decomposition, using the features obtained by encoding and multiplying the attenuation vectors obtained by decomposition, and then decoding, which can generate pure EEG signals and artifacts.

[0005] In addition, in recent years, the Transformer has been widely used in natural language processing, computer vision and other fields, and its attention mechanism is a key part of its internal structure design. The self-attention mechanism used by the Transformer has a quadratic computational complexity, which brings high computational complexity, high memory usage, and low efficiency in predicting long sequence outputs. The Transformer model is good at capturing global interactions based on content, but it is not good at extracting local features. Therefore, many studies have optimized its structure. Zhou et al. (Zhou H, Zhang S, Peng J, et al. Informer: Beyond efficient transformer for long sequence time-series forecasting [C] / / Proceedings of the AAAI conference on artificial intelligence. 2021, 35(12): 11106-11115) designed the Informer structure, which halved the output of each layer of the Transformer model's self-attention layer to highlight the main attention and reduce the computational complexity of the self-attention mechanism. W. Yu et al. (W. Yu, M. Luo, P. Zhou, C. Si, Y. Zhou, X. Wang, J. Feng, and S. Yan, “Metaformer is actually what you need for vision,” in Proceedings of the IEEE / CVF conference on computer vision and pattern recognition, 2022, pp. 10 819-10 829.) designed the Metaformer model, which verified that the multi-head self-attention module of the Transformer model can be replaced by a spatial MLP or PoolFormer while achieving better results with lower computational complexity. Gulati et al. (Gulati A, Qin J, Chiu C C, et al. Conformer: Convolution-augmented transformer for speech recognition [J]. arXiv preprint arXiv:2005.08100, 2020) designed the Conformer model, which considered the characteristics of the Transformer model's ability to capture global features and CNN's effective use of local features, combined convolutional neural networks and Transformers to model the local and global dependencies of sequence information, and achieved the best results.Liu et al. (Liu Z, Lin Y, Cao Y, et al. Swin transformer: Hierarchical vision transformer using shifted windows [C] / / Proceedings of the IEEE / CVF international conference on computer vision. 2021: 10012-10022) designed a Swin transformer model based on the ViT model, introducing a sliding window mechanism that allows the model to learn information across windows, while also allowing the model to save computational resources and focus on global and local information through downsampling layers. L. Chen et al. (L. Chen, X. Chu, X. Zhang, and J. Sun,“Simple baselines for image restoration,” arXiv preprint arXiv:2204.04676, 2022) improved the structure of the Transformer model, reducing computational complexity while maintaining effectiveness, using a module combining convolution and channel attention instead of a multi-head self-attention module. The GLU (Gated Linear Unit) was introduced to replace the RELU activation and simplify it, improving the model’s effectiveness.

[0006] It is worth noting that the U-Net structure has been proven to be effective for denoising, and combining it with the Transformer can further improve the denoising effect of the model (Z. Zhu, S. Zhang, L. Qiu, H. Wang and G. Luo, "Axis-based Transformer UNet for RGB remote sensing image denoising," in IEEE Signal Processing Letters, doi: 10.1109 / LSP.2024.3418717). Cao et al. (Cao, H. et al. (2023). Swin-Unet: Unet-Like Pure Transformer for Medical Image Segmentation. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds) Computer Vision-ECCV 2022 Workshops. ECCV 2022) designed a Swin-Unet model for image denoising: replacing the convolution in the Unet architecture with Swin-Transformer, which uses sliding windows, relative position encoding, and hierarchical structure. Global and local features are extracted. The model has better image denoising effect than advanced methods. Liu L et al. (Liu L, Guan H, Ma J, et al. A mask free neural network for monaural speech enhancement [J]. arXiv preprint arXiv:2306.04286, 2023) designed a speech denoising model MFNet, which uses STDCT to obtain real two-dimensional data. They also based on the U-Net architecture, replaced the ordinary convolution with the global and local feedforward module (GLFB) module and reduced the model parameter quantity, which has similar global and local modeling ability to the transformer. The speech denoising effect is good and the computational complexity is low.

[0007] Through the overview, it is found that most of these denoising methods use the reconstruction of the generated electroencephalogram, and there are few methods for separating potential features. Most of the model structures use convolutional neural networks, and only a few methods combine convolution with LSTM, Transformer and other sequence models. Previous studies have shown that the combination of convolution and sequence models can well extract and fuse local and global features. However, most of the existing researches have the problems of insufficient interaction extraction ability of global and local fine-grained features and high computational complexity of the model. SUMMARY

[0008] The present application aims at overcoming the deficiencies of the prior art, and provides a convolution attention-based adaptive separation network for electroencephalogram artifact removal.

[0009] The present application aims at overcoming the deficiencies of the prior art, and provides a convolution attention-based adaptive separation network for electroencephalogram artifact removal.

[0010] (1) The encoding module comprises an input module, three sub-modules A and three down-sampling modules, the three sub-modules A are sequentially connected, and each sub-module A is connected with a down-sampling module;

[0011] The input module uses a standard one-dimensional convolution to convert one-dimensional input data into two-dimensional data with two dimensions of sequence and channel, as shown in the following formula:

[0012] X = InputBlock (PE) (1)

[0013] PE represents the contaminated EEG signal, and X represents the output matrix converted by the input module; InputBlock represents the input module;

[0014] The encoding process of the sub-module A and the down-sampling module is represented as:

[0015] E1 = CABS (X) (2)

[0016] E2 = CABS (DownSampling (E1)) (3)

[0017] E3 = CABS (DownSampling (E2)) (4)

[0018] M1 = DownSampling (E3) (5)

[0019] E1, E2 and E3 respectively represent the encoding features output by the three sub-modules A, and M1 represents the latent feature; CABS represents the sub-module A, and DownSampling represents the down-sampling module;

[0020] (2) The separation module comprises: a sub-module B composed of a plurality of sequential convolution modules, a PRelu activation function, a point convolution module, and a sigmod activation function, wherein the sub-module B, the PRelu activation function, the point convolution module, and the sigmod activation function are sequentially connected; the latent feature M1 passes through these modules to obtain the weight of the feature related to the electroencephalogram signal, and then the weight is multiplied by the latent feature M1 to obtain a new latent feature M2, thereby realizing adaptive separation of the latent feature; the specific process is shown in the following formula:

[0021] F= PRelu(TCBS(M1)) (6)

[0022] W= Sigmoid(PointConv(F))) (7)

[0023] M2 = W*M1 (8)

[0024] W represents the weight of the feature related to the electroencephalogram signal, and TCBS represents the sub-module B and PointConv represents the point convolution;

[0025] (3) The decoding module comprises three sub-modules C, three up-sampling modules, and an output module, wherein the three sub-modules C are sequentially connected, an up-sampling module is connected before each sub-module C, and the output of the third sub-module C is connected to the output module; wherein the input of the first sub-module C is the combination of the output of the third sub-module A and the first up-sampling module, the input of the second sub-module C is the combination of the output of the second sub-module A and the second up-sampling module, and the input of the third sub-module C is the combination of the output of the first sub-module A and the third up-sampling module; the decoding process of the sub-module C and the up-sampling module is shown in the following formula:

[0026] M3 =UpSampling(M2) (9)

[0027] D1= UpSampling(CABS(M3+E3)) (10)

[0028] D2= UpSampling(CABS(D1+E2)) (11)

[0029] D3 = CABS(D2+E1) (12)

[0030] D1, D2, and D3 represent the decoding features in the decoding process, CABS represents the sub-module C, and UpSampling represents the up-sampling module;

[0031] The output module uses a standard one-dimensional convolution with a convolution kernel size of 3 to convert two-dimensional data with two dimensions of sequence and channel into one-dimensional reconstructed electroencephalogram data, thereby obtaining a pure EEG signal, as shown in the following formula:

[0032] CE = OutBlock(D3) (13)

[0033] CE stands for Refactored EEG, and OutBlock stands for Output Module.

[0034] The beneficial effects of this invention are as follows: Based on the U-net architecture, this invention adaptively separates EEG discriminative features by constraining latent features through a separation module. Furthermore, this invention improves upon the inherent weaknesses of the Transformer by designing a novel convolutional attention module to enhance the learning ability of coarse-grained and fine-grained feature interactions, while reducing the number of parameters and computational cost. Experimental results show that ASNet achieves excellent artifact removal performance on both semi-synthetic and real datasets. Moreover, ASNet has lower computational complexity compared to other methods, indicating that this method can efficiently extract clean EEG signals from contaminated raw EEG images, which is of great significance for the automatic and real-time preprocessing of EEG signals. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the adaptive separation network of the present invention;

[0036] Figure 2 This is a schematic diagram of the structure of submodule B in the adaptive separation network of the present invention;

[0037] Figure 3 This is a schematic diagram of the structure of the extended convolutional module in the adaptive separation network of the present invention;

[0038] Figure 4 This is a schematic diagram of the structure of submodule A and submodule C of the present invention;

[0039] Figure 5 This is a schematic diagram of the channel attention module in submodules A and C of the present invention;

[0040] Figure 6 This is a schematic diagram of the spatial attention module in submodules A and C of the present invention;

[0041] Figure 7 for Figure 4 Comparison of RRMSE and CC values ​​for EOG removal results;

[0042] Figure 8 Comparison of RRMSE and CC values ​​for EOG removal results under different signal-to-noise ratios;

[0043] Figure 9 Five waveforms showing the removal of Eog artifacts;

[0044] Figure 10The RRMSE and CC comparison chart of the EMG removal result;

[0045] Figure 11 The RRMSE and CC comparison chart of the EMG removal result under different signal-to-noise ratios;

[0046] Figure 12 The waveform chart of five EMG artifact removals. DETAILED DESCRIPTION

[0047] The application proposes an adaptive separation network (ASNet) based on convolution attention for electroencephalogram artifact removal. The network mainly includes three main functional blocks, which are an encoding module, a separation module and a decoding module. The encoding module includes an input module, three sub-modules combined by multiple convolution attention modules and three down-sampling modules, which abstract the input contaminated electroencephalogram into latent features. The separation module includes a sub-module combined by multiple time convolution modules, and the time convolution module is combined by multiple expansion convolution modules with gradually expanding kernel spacing. The weight of the correlation between the input latent features and the electroencephalogram signal features is obtained, and then multiplied to the latent features to realize the adaptive separation of the electroencephalogram discriminative features. The module has a gradually expanding receptive field, so that the convolution module can capture global features. The decoding module includes an output module, three sub-modules combined by multiple convolution attention modules and three up-sampling modules, which decode the latent features into reconstructed electroencephalogram. Among them, the application also improves the shortcomings of the Transformer itself, designs a novel convolution attention module to improve the fine-grained feature interaction learning ability, while reducing the parameter quantity and the calculation amount. And it is used as the main part of the encoder and decoder in the U-net architecture, so that the U-net architecture extracts and fuses features that take into account the local dependence and global time dependence of the electroencephalogram information.

[0048] The ASNet of the application is an end-to-end network that reconstructs the clean electroencephalogram signal from the original contaminated electroencephalogram signal by learning nonlinear transformation. The technical solutions of the application are further described below with reference to the accompanying drawings.

[0049] As shown in Figure 1 A convolution attention-based adaptive separation network for electroencephalogram artifact removal, according to the application, includes an encoding module, a separation module and a decoding module connected in sequence; the input of the network is a single-channel contaminated EEG signal, and the output is a pure EEG signal;

[0050] (1) The encoding module includes an input module, three sub-modules A and three down-sampling modules, the three sub-modules A are connected in sequence, and a down-sampling module is connected after each sub-module A;

[0051] The input module converts one-dimensional input data into two-dimensional data with two dimensions of sequence and channel using standard one-dimensional convolution, as shown in the following formula:

[0052] X = InputBlock (PE) (1)

[0053] PE represents the contaminated EEG signal, X represents the output matrix converted by the input module, InputBlock represents the input module, and the shape of the data variable is: PE ∈ R 1xS (S represents the sequence length), X ∈ R CxS (C represents the channel dimension), and R represents the real number field. The shape of the one-dimensional EEG data input is R S , the dimension is expanded to R 1xS , and then the contaminated EEG is input to the input module for processing to obtain two-dimensional data X.

[0054] The encoding process of the sub-module A and the down-sampling module is represented as:

[0055] E1 = CABS (X) (2)

[0056] E2 = CABS (DownSampling (E1)) (3)

[0057] E3 = CABS (DownSampling (E2)) (4)

[0058] M1 = DownSampling (E3) (5)

[0059] E1, E2, and E3 represent the encoding features output by the three sub-modules A, M1 represents the latent feature, CABS represents the sub-module A, and DownSampling represents the down-sampling module. The shape of the data variable is represented as: X ∈ R CxS , E1 ∈ R CxS , E2 ∈

[0060] R 2CxS / 2 , E3 ∈ R 4CxS / 4 , and M1 ∈ R 8CxS / 8 . As shown in formula (2), the feature matrix X is first transmitted to the sub-module to obtain E1; as shown in formula (3), E1 is input to the down-sampling module and then transmitted to the sub-module to obtain E2; as shown in formula (4), E2 is input to the down-sampling module and then transmitted to the sub-module to obtain E3; as shown in formula (5), E3 is transmitted to the down-sampling module to obtain the latent feature M1.

[0061] (2) The separation module comprises: a sub-module B composed of a plurality of sequential connection modules, a PRelu activation function, a point convolution module, and a sigmod activation function, wherein the sub-module B, the PRelu activation, the point convolution module, and the sigmod activation function are sequentially connected; the latent feature M1 passes through these modules to obtain the weight of the feature related to the electroencephalogram signal, and then the weight is multiplied with the latent feature M1 to obtain a new latent feature M2, thereby realizing adaptive separation of the latent feature; the specific process is shown in the following formula:

[0062] F = PRelu(TCBS(M1)) (6)

[0063] W = Sigmoid(PointConv(F)) (7)

[0064] M2 = W*M1 (8)

[0065] W represents the weight of the feature related to the electroencephalogram signal, TCBS represents the sub-module B, PRelu represents the PRelu activation function, Sigmoid represents the Sigmoid activation function, and PointConv represents the point convolution. The shape of the data variable is: M1∈R 8CxS / 8 , M2∈R 8CxS / 8 , F∈R 8CxS / 8 , W∈R 8CxS / 8 . As shown in formula (6), the latent feature M1 passes through the sub-module to extract the feature and is activated by PRelu to obtain F; as shown in formula (7), the feature F is point-convolved and activated by Sigmoid to obtain the weight W; as shown in formula (8), finally, the corresponding points of the weight matrix W and the latent feature matrix M1 are multiplied to obtain M2.

[0066] The sub-module B comprises a plurality of time series convolution modules, and the time series convolution module is sequentially connected and combined by a plurality of expansion convolution modules (DCB) with gradually expanding kernel spacing, as shown in Figure 2 , the kernel spacing increases exponentially; in the figure, the value of d represents the kernel spacing, the sequence dimension of the latent feature of the model design is 64, the one-dimensional kernel size is 3, and the maximum kernel spacing is 2 i , i is set to 4 in the model, that is, the maximum kernel spacing is 16, so as to ensure sufficient extraction of the context feature by the kernel and ensure that the extracted feature has long-distance time dependence of the electroencephalogram information.

[0067] As shown in Figure 3 , the expansion convolution module is composed of a depth separable convolution and a point convolution. The input is added to the output using a residual connection. The expansion convolution module is sequentially connected by a point-by-point convolution, a layer specification, a PRelu function, a depth expansion convolution, and a point-by-point convolution. The input data of the expansion convolution module and the output data of the point-by-point convolution are combined to serve as the output of the expansion convolution module.

[0068] (3) the decoding module comprises three sub-modules C, three up-sampling modules and an output module, the three sub-modules C are connected in sequence, one up-sampling module is connected before each sub-module C, and the output of the third sub-module C is connected with the output module; wherein the input of the first sub-module C is the combination of the output of the third sub-module A and the first up-sampling module, the input of the second sub-module C is the combination of the output of the second sub-module A and the second up-sampling module, and the input of the third sub-module C is the combination of the output of the first sub-module A and the third up-sampling module; the decoding processes of the sub-module C and the up-sampling module are shown in the following formulas:

[0069] M3 = UpSampling(M2) (9)

[0070] D1 = UpSampling(CABS(M3 + E3)) (10)

[0071] D2 = UpSampling(CABS(D1 + E2)) (11)

[0072] D3 = CABS(D2 + E1) (12)

[0073] D1, D2 and D3 respectively represent decoded features output in the decoding process, CABS represents the sub-module C, and UpSampling represents the up-sampling module; the shape of the data variable is: M2 ∈ R 8CxS / 8 , M3 ∈ R 4CxS / 4 , D1 ∈ R 2CxS / 2 , D2 ∈ R CxS , D3 ∈ R CxS . As shown in formula (9), the latent feature M2 is first transmitted into the up-sampling module to obtain M3; as shown in formula (10), M3 and E3 are added and then input into the sub-module to obtain D1 after being transmitted into the up-sampling module; as shown in formula (11), D1 and E2 are added and then input into the sub-module to obtain D2 after being transmitted into the up-sampling module; as shown in formula (12), D2 and E1 are added and then input into the sub-module to obtain D3.

[0074] The output module uses a standard one-dimensional convolution with a kernel size of 3 to convert two-dimensional data with two dimensions of sequence and channel into one-dimensional reconstructed electroencephalogram data, to obtain a pure EEG signal, as shown in the following formula:

[0075] CE = OutBlock(D3) (13)

[0076] CE represents a reconstructed EEG (Corrected EEG), and OutBlock represents the output module. D3 represents a decoded feature in the decoding process, and the shape of the data variable is: D3 ∈ R CxS , CE ∈ R 1xSD3 is input to the output module to obtain CE, and then the shape is R. 1xS The reconstructed EEG reduced its dimensionality to obtain a shape of R. S Reconstructing brainwaves.

[0077] Submodules A and C have the same structure, both containing multiple sequentially connected convolutional attention modules. The design of these convolutional attention modules is based on a Transformer architecture including a multi-head self-attention module and a feedforward network module. The self-attention mechanism used by the multi-head self-attention module has quadratic computational complexity, making it unsuitable for mobile devices or resource-constrained devices. It also suffers from insufficient extraction of local features. To address this issue, such as... Figure 4 As shown, this invention uses a combination of depthwise separable convolutions and attention modules to replace the multi-head attention module. It includes sequentially connected layer canonical A, pointwise convolution A, depthwise convolution, gate activation function A, channel attention module, spatial attention module, and pointwise convolution B. Two pointwise convolutions replace the feedforward layer, connected after the combination of depthwise separable convolutions and attention modules, and include sequentially connected layer canonical B, pointwise convolution C, gate activation function B, and pointwise convolution D. The input of layer canonical B and the input of layer canonical A are combined with the output of pointwise convolution B, and the input of layer canonical B and the output of pointwise convolution D are combined to serve as the output of the sub-module. Using a combination of depthwise separable convolutions and attention modules to replace the multi-head attention module, and using two pointwise convolutions to replace the feedforward layer, ensures effective extraction of global and local features while achieving model lightweighting, reducing the number of parameters and computational cost.

[0078] Depthwise separable convolution consists of pointwise convolution and depthwise convolution. Pointwise convolution is used to extract channel features, aggregating all channels to increase or decrease feature dimensionality. Depthwise convolution is used to extract spatial features, grouping convolutions along the feature dimension and performing independent depthwise convolutions on each channel. Compared to standard convolution, it achieves the same feature extraction effect but with lower computational cost and fewer parameters.

[0079] like Figure 5 As shown, the channel attention module of the present invention consists of sequentially connected global average pooling (GAP), point convolution, and sigmoid activation function. Global average pooling is performed on the input feature map to obtain one-dimensional channel information. The one-dimensional channel information is then processed by point convolution and sigmoid activation function to obtain channel attention weights. The channel attention weights are then multiplied with the input feature map to learn the channel relationship between convolutional features, which is used as the output of the channel attention module.

[0080] like Figure 6As shown, the spatial attention module is used to learn the spatial relationship between convolutional features, including a global maximum pooling (GMP), a global average pooling (GAP), a splicing layer, a point convolution module and a Sigmoid activation function connected in sequence; the spatial attention module respectively performs global maximum pooling and global average pooling on the data output by the channel attention module, so that features with more useful information are concentrated and features without information are filtered, generating two feature maps representing different information; the two feature maps are spliced and merged through the splicing layer and then input into the point convolution module for feature fusion, and finally the Sigmoid operation is performed to generate a weight map, which is then multiplied with the input feature map of the spatial attention module as the output of the spatial attention module, so that the target region is enhanced. The combination of the channel attention module and the spatial attention module can learn the channel and spatial relationship of the convolutional features. The global and local features are integrated.

[0081] The technical effects of the present application are further verified by specific embodiments below.

[0082] 1. Training and testing: The contaminated electroencephalogram signal is used as input, and the reconstructed electroencephalogram signal is output to train the model in a supervised manner. The mean square error (MSE) between the output and the expected reconstructed clean EEG is used as the loss function. In order to train the network, the Adam optimization algorithm is used, and the description of the hyperparameters of the model is shown in Table 1(a), and the setting of the hyperparameters is shown in Table 1(b). After mixing the electroencephalogram data and artifacts of the semi-synthetic data set, the training set, the validation set and the test set are divided in the ratio of 8:1:1. After training, the model can be used to remove the electrooculogram artifact and the electromyogram artifact in the single-channel electroencephalogram. The effect of electroencephalogram denoising may affect the subsequent electroencephalogram tasks such as event-related potential (ERP) analysis and electroencephalogram source localization, and the model will be tested on the semi-synthetic data set and the real data set.

[0083] Table 1(a)

[0084] Symbol Description N1 Number of convolutional attention blocks (CABS) included in the sub-module N2 Number of temporal convolution blocks (TCBS) included in the sub-module B Batch size S Length of the input EEG data K Size of the convolution kernel I Initial learning rate

[0085] Table 1(b)

[0086] N1 N2 B S K I 8 8 50 512 3 0.0005

[0087] 2. Data set

[0088] (1) Semi-synthetic dataset: EEGdenoiseNet (Zhang H, Zhao M, Wei C, et al. EEGdenoiseNet: a benchmark dataset for deep learning solutions of EEG denoising [J]. Journal of Neural Engineering, 2021, 18(5): 056057) designed a semi-synthetic dataset for EEG artifact removal, which contains 4514 clean EEG segments, 3400 eye artifact segments and 5598 muscle artifact segments. Each EEG signal segment has 512 samples (2s), and the clean EEG signal segment is linearly mixed with the EOG or EMG artifact segment to generate the contaminated EEG signal according to formula (2). The signal-to-noise ratio (formula (3)) is from -7 to 2 dB. The semi-synthetic dataset of contaminated EEG obtained by mixing clean EEG and artifacts is used to train the model.

[0089] y = x + l·n (14)

[0090]

[0091] (2) Real dataset: BCI Competition 2008 Graz Dataset A (Brunner C, Leeb R, Müller-Putz G, et al. BCI Competition 2008-Graz data set A [J]. Institute for knowledge discovery (laboratory of brain-computer interfaces), Graz University of Technology, 2008, 16: 1-6) covers EEG data of 9 participants in a finger-based BCI study, involving four motor imagery tasks: left hand, right hand, both feet and tongue, divided into four categories. Each participant was tested twice, each time containing 6 rounds, 48 attempts, a total of 288 times. Five minutes of EEG data were collected before the test to evaluate the EOG effect, including open eyes, closed eyes and eye movement. Data were recorded through 22 10-20 system electrodes, with a sampling rate of 250 Hz, band-pass filtering of 0.5-100 Hz, and 50 Hz notch filtering. Three EOG channels were recorded simultaneously to assist in removing artifacts. The data were checked by experts to exclude bad trials, stored in GDF format, with detailed labels, for BCI performance evaluation.

[0092] (3) Evaluation index: Two quantitative performance indicators, relative root mean square error (RRMSE) and correlation coefficient (CC), are used to evaluate the model.

[0093] 1) RRMSE: RRMSE is a normalized expression of RMSE using actual values as scaling terms in its calculation.

[0094] In the context of removing EEG signal artifacts, RRMSE can measure the deviation between clean and contaminated signals, and its calculation method is as follows:

[0095]

[0096] where S = {s k k = 1, 2, …, K} represents a clean EEG signal segment with a sequence length of K, is the corresponding modified segment of S. When the RRMSE value decreases, the performance of the artifact removal method improves.

[0097] 2) CC: CC can measure the similarity between clean and contaminated EEG segments, indicating the degree to which the corrected EEG segment contains effective information from the clean EEG segment. The calculation of CC can be defined as:

[0098]

[0099] where Cov and Var represent the calculation of covariance and variance, respectively. It is worth noting that the higher the CC, the more information the corrected EEG segment contains.

[0100] 3. Experiment and results

[0101] (1) Results of semi-synthetic EEG data

[0102] This experiment first uses the semi-synthetic EEG dataset to test the denoising performance of the model, and compares the results with EEGDNet, Denoiseformer and DeepSeparator.

[0103] 1) EOG artifact removal: This experiment uses two indicators to quantify the average performance of EOG artifact removal for more than 5000 test samples, namely relative root mean square error (RRMSE) and correlation coefficient (CC). As shown in Table 2, ASNet achieved the lowest RRMSE, and the highest CC and lower standard deviation, which indicates that ASNet is superior to other deep learning methods in EOG artifact removal. In this embodiment, the RRMSE of the EOG removal result is shown in Figure 7 (a), and the CC is shown in Figure 7 (b). From Figure 7The box plots in FIG. 6 show that ASNet has a more concentrated distribution of RRMSE, CC, and correlation coefficient than other models, which indicates that ASNet is more robust in EOG artifact removal. To further investigate the robustness of the model in denoising, the performance of the model under various signal-to-noise ratios (SNRs) (from -7 to 2 dB in formula (3)) is tested. The RRMSE and CC of the EOG removal results under different SNRs are shown in FIGS. 6(a) and 6(b), respectively. Generally, the performance of all four methods deteriorates as the SNR decreases, but ASNet is indeed superior to other methods at lower RRMSE and higher CC SNR levels. The results show that ASNet is robust even when there are large EOG artifacts in the input. In Figure 8 FIGS. 6(a) and 6(b) show the box plots of the RRMSE and CC of the EOG removal results, respectively. Figure 9 FIG. 7 shows five waveforms of EOG artifact removal, including the contaminated EEG signal, the clean EEG signal, and the reconstructed EEG signal. Comparing the waveforms of the clean EEG signal and the reconstructed EEG signal, they are basically coincident, which indicates that the ASNet model can well remove the EOG artifact and retain the clean EEG signal.

[0104] Table 2 Comparison of EOG removal results

[0105] Metric EEGDNet Denoiseformer DeepSeparator ASNet RRMSE 0.384 0.292 0.612 0.252 CC 0.912 0.951 0.769 0.966

[0106] 2) EMG artifact removal: The performance of EMG artifact removal is quantified in Table 3. The results show that the ASNet model is much better than other competitive models in both relative root mean square error (RRMSE) and correlation coefficient (CC). Figure 10 FIGS. 8(a) and 8(b) are RRMSE and CC plots of the EMG removal results, respectively, Figure 10 The box plots in FIG. 8 also show the robust performance of ASNet. In Figure 11 In FIGS. 8(a) and 8(b), the RRMSE and CC of the EMG removal results under different SNRs are compared, respectively, Figure 11 The EMG artifact removal performance under different SNR levels is quantitatively evaluated, and the results are slightly different from those of EOG, while ASNet is still the best model. In Figure 12 FIG. 9 shows five waveforms of EMG artifact removal. Since the EMG artifact is more complex than the EOG artifact, the EMG artifact removal is less effective than the EOG artifact removal, but the waveforms of the clean EEG signal and the reconstructed EEG signal can still be basically coincident, which indicates that the ASNet model can well remove the EMG artifact and retain the clean EEG signal.

[0107] Table 3 Comparison of EMG removal results

[0108] Metric EEGDNet Denoiseformer DeepSeparator ASNet RRMSE 0.624 0.372 0.712 0.328 CC 0.691 0.894 0.634 0.915

[0109] (2) Real dataset results: In this experiment, BCI Competition 2008 Graz Dataset A was used for training and testing of motor imagery four-classification. Ang (Ang, Kai & Chin, Zhang & Zhang, Haihong & Guan, Cuntai. (2008). Filter Bank Common Spatial Pattern (FBCSP) in brain-computer interface. Proceedings of the International Joint Conference on Neural Networks. 2390-2397. 10.1109 / IJCNN.2008.4634130) and others proposed a novel filter bank common spatial pattern (FBCSP) method that can automatically select key spatiotemporal discriminative EEG features. Sherwani (Sherwani, Fahad & Shanta, Shahnoor & Ksm Kader Ibrahim, Babul Salam & Huq, Saif. (2016). Wavelet based feature extraction for classification of motor imagery signals. 360-364. 10.1109 / IECBES.2016.7843474) and others combined wavelet transform analysis and neural network (NN) EEG classification technology. Wavelet decomposition was used to decompose the recorded EEG signal into four levels. The data was filtered, useless channels were deleted, and segmented. Then add a certain amount of noise, use the ASNet model for denoising and non-denoising for comparison. The common spatial pattern (FBCSP) method and wavelet transform were used to extract the motor imagery features of the EEG, and then the Sequential model in the Keras classifier model was used for classification. Finally, the test was conducted. As shown in Table 4, the effect of motor imagery four-classification was improved after using the ASNet model for denoising, which means that the denoising effect of ASNet improves the subsequent classification task of the real dataset.

[0110] Table 4 Motor imagery experiment results

[0111] Accuracy Kappa Precision Recall Before Denoising 0.630 0.504 0.645 0.631 After Denoising 0.636 0.512 0.658 0.636

[0112] (3) Ablation experiment: This experiment ablates the gate linear unit (GLU), channel attention and spatial attention (CA&SA), and separator. As shown in Table 5, removing the gate linear unit module has a small effect on the ASNet model, and the correlation coefficient index of the removal effect of EOG artifacts and EMG artifacts decreases by 0.01 and 0.034, respectively. The gate linear unit has a certain improvement on the model. The influence of removing the channel attention and spatial attention is greater than removing the gate linear unit module, and the correlation coefficient index of the removal effect of EOG artifacts and EMG artifacts decreases by 0.023 and 0.044, respectively, indicating that the channel attention and spatial attention enhance feature extraction. The separator has the greatest influence on the model, and the model performance decreases significantly after removing the separator module, and the correlation coefficient index of the removal effect of EOG artifacts and EMG artifacts decreases by 0.039 and 0.072, respectively. Thus, the importance of separating latent features is shown.

[0113] Table 5 Ablation experiment results

[0114] GLU CA&SA Separator EOG EMG √ √ 0.956 0.881 √ √ 0.943 0.871 √ √ 0.927 0.843

[0115] (4) Model parameters and inference time: The sub-modules (CABS) in the encoder and decoder include multiple convolutional attention modules, and the separator uses a combination of multiple temporal convolution modules (TCBS). The number of modules included in CABS and TCBS affects the model effect. With the repeated superposition of small modules in the module, the model can be improved, but there is an upper limit. As shown in Table 6, when the number of modules is 8, the model achieves the best performance in removing EOG and EMG.

[0116] Table 6 Parameter experiment results

[0117] Number of modules EOG EMG 8 0.966 0.915 2 0.954 0.901 3 0.958 0.908 16 0.960 0.912

[0118] In addition to two indicators (relative root mean square error and correlation coefficient), the parameter amount and inference time of the model are also important to measure the effectiveness of the electroencephalogram denoising model, especially for portable electroencephalogram devices, which not only need effective denoising but also need to be real-time. As shown in Table 7, compared with the Denoiseformer model with similar effects, the ASNet model has a lower parameter amount and less inference time, and the improvement of the Transformer model reduces the parameter amount and computational amount.

[0119] Table 7 Parameter amount and inference time comparison

[0120] Method Parameter (million) Inference time (ms) EEGDNet 0.18 1.97 Denoiseformer 8.06 24.17 DeepSeparator 0.68 2.75 ASNet 5.48 4

[0121] In the present application, a deep learning model ASNet is proposed, which can extract electroencephalogram signals from contaminated EEG data. Removing artifacts mixed in electroencephalogram signals is an important step for subsequent electroencephalogram signal analysis. Compared with previous deep learning methods, ASNet can achieve satisfactory performance when processing single-channel electroencephalogram signal data containing electrooculogram artifacts or electromyogram artifacts. The strong denoising effect of ASNet provides great potential for the application of EEG denoising. In addition, the performance of ASNet at different signal-to-noise ratio levels is reliable, which makes ASNet can be used in data with different noise environments. Once ASNet is trained for application, its denoising ability becomes fast and automatic. It can be further combined with artificial intelligence hardware to realize real-time electroencephalogram denoising, which will greatly promote electroencephalogram-based brain-computer interface applications, ERP analysis and source localization.

[0122] Another advantage of ASNet compared with other deep learning models is its low parameter amount and computational amount. In order to solve the problem of large computational amount of the multi-head self-attention module of Transformer, the MobileNet module with attention mechanism is used to replace the multi-head attention module, and the deep separable convolution used in MobileNet module can greatly reduce the computational amount while ensuring the feature extraction effect. It is conducive to the lightweight of the module, and reduces the parameter amount and computational amount, as shown in Table 6.

[0123] Those skilled in the art will realize that the embodiments described herein are for the purpose of helping the reader understand the principles of the present application, and should be understood as not limiting the scope of protection of the present application to such specific statements and embodiments. Those skilled in the art can make various other specific modifications and combinations according to the technical inspiration disclosed in the present application without departing from the essence of the present application, and these modifications and combinations are still within the scope of protection of the present application.

Claims

1. An adaptive separation network based on convolutional attention for electroencephalographic artifact removal, characterized in that, The network comprises sequentially connected encoding modules, separation modules and decoding modules; the input of the network is a single-channel contaminated EEG signal, and the output is a pure EEG signal; (1) The encoding module comprises an input module, three sub-modules A and three down-sampling modules, the three sub-modules A are sequentially connected, and a down-sampling module is connected after each sub-module A; The input module converts one-dimensional input data into two-dimensional data with two dimensions of sequence and channel by using standard one-dimensional convolution, as shown in the following formula: X = InputBlock (PE) (1) PE represents the contaminated EEG signal, and X represents the output matrix of the input module after conversion; InputBlock represents the input module; The encoding process of the sub-module A and the down-sampling module is as follows: E1 = CABS (X) (2) E2 = CABS (DownSampling (E1)) (3) E3 = CABS (DownSampling (E2)) (4) M1 = (E3) (5) E1, E2 and E3 represent the encoding features output by the three sub-modules A respectively, M1 represents a latent feature, CABS represents the sub-module A, and DownSampling represents the down-sampling module; (2) The separation module comprises a sub-module B composed of sequentially connected multiple time convolution modules, a PRelu activation function, a point convolution module and a sigmod activation function, the sub-module B, the PRelu activation function, the point convolution module and the sigmod activation function are sequentially connected; the latent feature M1 passes through these modules to obtain a weight of a feature related to the electroencephalogram signal, and then the weight is multiplied by the latent feature M1 to obtain a new latent feature M2, thereby realizing adaptive separation of the latent feature; the specific process is shown in the following formula: F = PRelu (TCBS (M1)) (6) W = Sigmoid (PointConv (F)) (7) M2 = W * M1 (8) W represents the weight of the feature related to the electroencephalogram signal, TCBS represents the sub-module B, and PointConv represents the point convolution; (3) The decoding module comprises three sub-modules C, three up-sampling modules and an output module, the three sub-modules C are sequentially connected, an up-sampling module is connected before each sub-module C, and the output of the third sub-module C is connected with the output module; wherein the input of the first sub-module C is a combination of the output of the third sub-module A and the first up-sampling module, the input of the second sub-module C is a combination of the output of the second sub-module A and the second up-sampling module, and the input of the third sub-module C is a combination of the output of the first sub-module A and the third up-sampling module; the decoding process of the sub-module C and the up-sampling module is as follows: M3 = UpSampling (M2) (9) D1 = UpSampling (CABS (M3 + E3)) (10) D2 = UpSampling (CABS (D1 + E2)) (11) D3 = CABS (D2 + E1) (12) CABS represents a sub-module C, and UpSampling represents an up-sampling module; The output module uses a standard one-dimensional convolution with a kernel size of 3 to convert two-dimensional data with two dimensions of sequence and channel into one-dimensional reconstructed electroencephalogram data, so as to obtain a pure EEG signal, as shown in the following formula: CE = OutBlock(D3) (13) CE represents reconstructed EEG, and OutBlock represents an output module. The sub-modules A and C have the same structure and each contain multiple convolution attention modules connected in sequence, and the convolution attention module contains layer specification A, point-wise convolution A, deep convolution, gate activation function A, channel attention module, spatial attention module, point-wise convolution B, layer specification B, point-wise convolution C, gate activation function B and point-wise convolution D connected in sequence, the input of layer specification B and the combination of the input of layer specification A and the output of point-wise convolution B, and the combination of the input of layer specification B and the output of point-wise convolution D are taken as the output of the sub-module.

2. The convolution attention based adaptive separation network for electroencephalogram artifact removal according to claim 1, wherein, The channel attention module is composed of global average pooling, point convolution and Sigmoid activation function connected in sequence; the global average pooling performs global average pooling on the input feature map to obtain one-dimensional channel information, the one-dimensional channel information is subjected to point convolution and Sigmoid activation function to obtain channel attention weight, and then the channel attention weight is multiplied by the input feature map as the output of the channel attention module.

3. The adaptive separation network based on convolutional attention for electroencephalographic artifact removal according to claim 1, wherein, The spatial attention module is used for learning the spatial relationship between convolution features and includes global maximum pooling, global average pooling, a concatenation layer, a point convolution module and a Sigmoid activation function connected in sequence; the spatial attention module performs global maximum pooling and global average pooling on the data output by the channel attention module to generate two feature maps representing different information; the two feature maps are spliced and combined through the concatenation layer and then input into the point convolution module for feature fusion, and finally the weight map is generated through the Sigmoid operation, and then the weight map is multiplied by the input feature map of the spatial attention module as the output of the spatial attention module.

4. The adaptive separation network based on convolutional attention for electroencephalographic artifact removal according to claim 1, wherein, The sub-module B contains multiple time series convolution modules, and the time series convolution module is composed of multiple expansion convolution modules with gradually increasing convolution kernel spacing connected in sequence, and the convolution kernel spacing increases exponentially; the expansion convolution module is composed of point-wise convolution, layer specification, PReLU function, deep expansion convolution and point-wise convolution connected in sequence, and the input data of the expansion convolution module and the output data of the point-wise convolution are combined as the output of the expansion convolution module.

Images