Motor imagery electroencephalogram recognition method and system based on Kans-Transform

Abstract

The invention provides a motor imagery electroencephalogram (MI-EEG) recognition method and a motor imagery electroencephalogram (MI-EEG) recognition system based on Kans-Transform, and aims to effectively improve classification accuracy and robustness. The method is realized through four key modules: a preprocessing module, a time domain and space domain feature extraction module, a Kans-Transform module and a classification module. The preprocessing module enhances data diversity and model generalization ability through filtering and data enhancement technologies. The time domain and spatial domain feature extraction module extracts time domain and spatial domain features of EEG by using a deep convolutional network, introduces an efficient channel attention (ECA) module after time domain convolution to perform channel attention weighting, and obtains more comprehensive feature representation through feature fusion. According to the Kis-Transform module, Kolmogorov-Arnold Networks are used for replacing the traditional MLPs, and the feature capturing capability is enhanced. And finally, performing classification through a classification module. The method still has excellent performance under the condition of limited labeled data, is suitable for brain-computer interfaces, and has important application value in the fields of rehabilitation medicine and human-computer interaction.

Description

Technical Field

[0001] The present invention relates to the technical field of motor imagery electroencephalogram (EEG) signal processing, and in particular to a motor imagery electroencephalogram (EEG) signal recognition method and system based on Kans-Transformer. Background Art

[0002] Electroencephalogram (EEG) analysis technology plays an increasingly important role in neuroscience, clinical diagnosis, and brain-computer interface (BCI). Traditionally, EEG analysis relies on manual feature extraction and classical machine learning algorithms, such as power spectral density (PSD) and differential entropy (DE), which not only require a lot of expertise but also consume huge computing resources. With the development of deep learning technology, especially the application of convolutional neural networks (CNN) and Transformer architectures, automatic feature learning and the capture of long-term dependencies have become possible, which has promoted deep learning-based EEG signal recognition to become a new research trend.

[0003] The application of the Transformer model in EEG signal processing is gradually showing its unique advantages. Through the self-attention mechanism and carefully designed structure, the model can effectively capture the long-distance dependencies in EEG signals, which is crucial for a deep understanding of the complex dynamic activities of the brain. The parallel computing characteristics of the Transformer model significantly improve the speed of processing EEG signals, which is especially important for application scenarios that require real-time feedback. In addition, the high performance of the Transformer model enables it to perform well in a variety of EEG signal processing tasks, such as EEG signal decoding and visualization.

[0004] However, the Transformer model also faces some challenges when processing EEG signals. For example, the high dimensionality and complexity of EEG signals make it difficult for traditional Transformer models to effectively capture all important features, especially in terms of local features and nonlinear dynamic characteristics. In addition, the Transformer model requires a lot of computing resources and memory, which may become a limiting factor when processing large-scale EEG datasets.

[0005] Kans is a new type of neural network structure with learnable activation functions on the edges ("weights"), replacing the fixed activation function nodes in traditional multi-layer perceptrons (MLPs). The advantage of this structure is that it can better simulate the complex behavior of dynamic systems and enhance the model's ability to capture features. Studies have shown that Kans is more effective than MLPs in science-related tasks, such as fitting physical equations and solving partial differential equations.

[0006] Therefore, this patent proposes a Kans-Transformer-based EEG signal recognition method and system, aiming to achieve more accurate and efficient recognition of EEG signals by combining Kans' dynamic system modeling capabilities and the self-attention mechanism of the Transformer architecture. Summary of the invention

[0007] 1. Technical issues to be resolved

[0008] In view of the deficiencies of the prior art, the present invention provides a method and system for electroencephalogram (EEG) signal recognition based on Kans-Transformer, which solves the problems raised in the above-mentioned background technology.

[0009] (II) Technical solution

[0010] In order to achieve the above-mentioned purpose, the present invention specifically adopts the following technical scheme: a method and system for recognizing electroencephalogram signals based on Kans-Transformer, the specific steps are as follows:

[0011] Step 1: First obtain the original EEG data and the EEG data to be decoded, and preprocess them: filtering and data enhancement.

[0012] Step 2: Use a deep convolutional network to extract the temporal and spatial features of the EEG signal, and introduce the ECA module after the temporal convolution to perform channel attention weighting to enhance the expression of important features. At the same time, the feature fusion technology is used to integrate the temporal feature information of different scales to further improve the richness and discrimination ability of the features.

[0013] Step 3: Introduce Kolmogorov-Arnold Networks (Kans) into the Transformer Encoder module to improve the model's ability to understand and capture EEG signal features.

[0014] Step 4: Design a classifier including convolutional layers, normalization layers, activation function layers, and linear layers. The classifier receives the features processed by the Kans-Transformer module and outputs the classification results of left hand, right hand, feet, and tongue movement imagination. During the training process, the model parameters are adjusted through the back propagation algorithm, and the cross entropy loss function is used as the optimization objective function.

[0015] The step 1 comprises the following sub-steps:

[0016] Step 101: Use a multi-stage Butterworth filter to perform a first filtering process (high-pass and low-pass filtering process) on the MI-EEG signal data to obtain EEG data within the relevant range of the motor imagery stage (8-30 Hz), thereby effectively eliminating high-frequency noise interference.

[0017] Step 102: using a notch filter to perform a second filtering process on the MI-EEG signal data to eliminate power frequency interference;

[0018] Step 103: Implement signal segment reorganization data enhancement technology to reduce the impact of noise and artifact signals while improving the generalization ability of the model.

[0019] The step 2 comprises the following sub-steps:

[0020] Step 201: Perform multi-scale temporal convolution operations on the pre-processed MI-EEG signal to extract time domain features of different scales. These convolution operations can capture information at different time levels in the signal.

[0021] Step 202: Fuse the features extracted from time convolutions of different scales to form a richer and more comprehensive time domain feature representation.

[0022] Step 203: Based on the fused time domain features, the ECA module is used to perform channel attention weighting. The ECA module can automatically learn the importance of each channel, thereby strengthening the expression of important features.

[0023] Step 204: Based on the time domain features and the channel-weighted features, spatial domain features are further extracted through spatial domain convolution to capture the structural information of the signal in the spatial domain.

[0024] The step 3 comprises the following sub-steps:

[0025] Step 301: First, use a multi-head attention mechanism to improve the model's ability to learn the correlation between features;

[0026] Step 302: Integrate Kans in the Transformer Encoder module to enhance the deep understanding and capture of EEG signal features;

[0027] Step 303: Use layer normalization to normalize the output of each sublayer (multi-head attention layer and Kans layer), and add a residual connection after each sublayer to ensure effective information transmission and stable convergence during training.

[0028] The step 4 comprises the following sub-steps:

[0029] Step 401: Design a classifier structure consisting of a convolution layer, a normalization layer, an activation function layer and a linear layer, which is used to receive the output of the feature extraction module as the input of the classifier and output the final classification result label.

[0030] Step 402: During the training process, the classifier receives the spatiotemporal features output by the feature extraction module, uses the cross entropy loss function to calculate the difference between the predicted label and the true label, and updates the model parameters according to the loss value through the back propagation algorithm. The learning rate is initially set to 0.001. When the validation loss does not improve within 5 consecutive epochs, the learning rate is reduced to 0.1 times the original value. In addition, L2 regularization is used to prevent overfitting, and the early stopping method is used to stop training when the validation set accuracy does not improve for 10 consecutive epochs, and finally output the classification result.

[0031] (III) Beneficial effects

[0032] Compared with the prior art, the present invention provides a method and system for EEG signal recognition based on Kans-Transformer, which has the following beneficial effects:

[0033] The present invention effectively addresses the challenges brought by the high dimensionality and complexity of EEG signals through the deep fusion of deep convolutional networks and Kans-Transformer modules, especially in capturing local features and nonlinear dynamic characteristics, significantly improving the model's ability to capture and classify comprehensive features of EEG signals. In addition, while maintaining the performance of the model, the present invention reduces the consumption of computing resources and memory requirements by optimizing the Kans-Transformer structure, making it more efficient and economical when processing large-scale EEG data sets. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 A flow chart of a motor imagery EEG recognition method based on Kans-Transformer provided by the present invention.

[0035] Figure 2 This is a schematic diagram of the network structure of motor imagery EEG recognition based on the Kans-Transformer model of the present invention;

[0036] Figure 3 This is a schematic diagram of the structure of the temporal and spatial feature extraction module of the present invention;

[0037] Figure 4 This is a schematic diagram of the ECA module structure of the present invention;

[0038] Figure 5 Schematic diagram of the self-attention mechanism structure of the present invention.

[0039] Figure 6 This is a graph showing the decoding accuracy results of multiple subjects on the BCI IV 2a dataset according to an embodiment of the present invention.

[0040] Figure 7 is a training loss and accuracy curve diagram of some subjects in the embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example

[0043] An embodiment of the present invention proposes a method and system for motor imagery EEG recognition based on Kans-Transformer, which will be further described below with reference to the accompanying drawings in the specification to make the present invention clearer and easier to understand. The technical solution provided by the present invention mainly includes steps such as data preprocessing, time and space feature extraction, Kans-Transformer module processing and classification module, and each step is closely connected to jointly realize accurate recognition of motor imagery EEG signals.

[0044] The specific steps are as follows:

[0045] Step 1: Reference Figure 2 , obtain motor imagery EEG signals and perform preprocessing.

[0046] Furthermore, step 1 is as follows:

[0047] Step 101: Obtain the original EEG data and the EEG data to be decoded. This embodiment uses the EEG signal dataset of the left and right hand motor imagery of the Fourth International Brain-Computer Interface Competition in 2008 BCI competition IV dataset2a, with a sampling frequency of 250Hz, and collects EEG signals in various situations such as left hand, right hand, feet and tongue motor imagery.

[0048] Step 102: A fifth-order Butterworth bandpass filter of 8-30 Hz is used to filter the signal. By accurately setting the cutoff frequency, the frequency components outside the frequency band are effectively attenuated, thereby significantly improving the signal-to-noise ratio.

[0049] Step 103: Filtering is performed using a 50 Hz notch filter. After careful adjustment of its parameters, the notch filter forms a notch at 50 Hz, effectively eliminating this frequency interference and providing a purer signal basis for subsequent signal processing.

[0050] Step 104: Data enhancement is achieved through signal segment recombination. Signal segment recombination divides the original signal into several segments, and then randomly selects and reassembles them in order, effectively expanding the data set and improving the generalization ability of the model.

[0051] Specifically, the original signal is S = [s 1 ,s 2 ,…,s 1000 ], then the signal after segmentation is S = [S 1 ,S 2 ,…,S 10 ](S i =[s (i-1)×100+1 ,s (i-1)×100+2 ,…,s i×100 ]), then the new signal after random recombination is

[0052] (j 1 ,j 2 ,…,j 10 is a random sequence from 1 to 10). This process can effectively expand the data set and improve the generalization ability of the model.

[0053] Step 2: Reference Figure 3 ,Construct the temporal and spatial feature extraction modules, which mainly include ,temporal convolution module, ECA module and spatial convolution module.

[0054] Furthermore, step 2 is as follows:

[0055] Step 201: Use three convolution kernels of different sizes (1,11), (1,31) and (1,51) to perform convolution operations in the time dimension. In this way, time domain features of different scales can be extracted, thereby capturing information at different time levels in the signal.

[0056] Step 202: A splicing operation may be performed on the channel dimension of the feature map, that is, the features extracted at each scale are arranged in sequence in the channel direction to form a richer and more comprehensive time domain feature representation, so that subsequent processing can comprehensively consider information at different time scales.

[0057] Step 203: Reference Figure 4 , construct the ECA module, including the global average pooling layer, the 1D convolution layer, the sigmoid activation function layer and the Expand layer.

[0058] Specifically, first, the input feature map is processed by the global average pooling layer, and the feature values ​​of each channel are averaged in the spatial dimension to obtain a channel descriptor with a dimension of 1x1xC (C is the number of channels). The global average pooling formula is:

[0059]

[0060] Among them, H is the height of the feature map and W is the width of the feature map.

[0061] Then, the channel descriptor is transformed using one-dimensional convolution. Finally, the transformed result is passed through the sigmoid activation function to obtain the channel attention weight, and the weight is multiplied by the input feature map through the Expand layer to enhance the expression of important features.

[0062] Step 204: Based on the time domain features and the features after channel weighting, the spatial domain features are further extracted through spatial convolution, which effectively captures the structural information of the signal in the spatial domain, accurately reflects the correlation between different electrodes, and provides richer feature basis for subsequent classification and recognition.

[0063] Step 3: Reference Figure 5 , build the Kans-Transformer module, which mainly includes the multi-head attention mechanism and the Kans network module.

[0064] Furthermore, step 3 is as follows:

[0065] Step 301: Construct a multi-head self-attention mechanism. The multi-head self-attention mechanism extends the self-attention mechanism by introducing multiple parallel attention heads. Each head has its own set of parameters for calculating attention weights. In this way, the model can focus on different subspaces of the sequence in parallel, and each head can learn different feature representations.

[0066] The specific calculation process is as follows:

[0067] First, the input features are compared with the weight matrices of the three science departments Multiply them together to get the query vector Key Vector Sum value vector

[0068] Then calculate the attention weights:

[0069]

[0070] The Softmax function is used to normalize the attention score, and the formula is as follows:

[0071]

[0072] Finally, the attention outputs of each head are concatenated and a linear transformation is performed to obtain the final multi-head attention output.

[0073] Step 302: Construct the GR-Kan network module. Kans has a learnable activation function on the edge, and its structure can better simulate the complex behavior of dynamic systems. During the calculation process, Kans further processes the features through its unique calculation method based on the input features and learnable parameters, which can enhance the deep understanding and capture of EEG signal features.

[0074] Step 303: Use layer normalization to normalize the output of each sublayer (multi-head attention layer and Kans layer), and add a residual connection after each sublayer to ensure effective information transmission and stable convergence during training.

[0075] Step 4: Construct a classification module, which mainly consists of a convolutional layer, a batch normalization layer, an activation function layer, and a fully connected layer.

[0076] Step 401: First, a special convolutional layer is used to adjust the feature dimension to facilitate subsequent feature fusion; second, a batch normalization layer is introduced to speed up the training and improve the model stability; then, the activation function is used to introduce nonlinear elements to enhance the expression ability of the model; finally, the relationship between the feature and the weight matrix and the bias vector is calculated through the fully connected layer, and the classification result label is obtained through probability calculation.

[0077] Step 402: During the training process, the classifier receives the spatiotemporal features output by the feature extraction module and uses the cross entropy loss function to calculate the difference between the predicted label and the true label. The value of the loss function is gradient calculated by the back propagation algorithm to update the parameters of the model. The optimizer uses the Adam optimizer to adjust the model parameters, and the initial learning rate is set to 0.001. In order to further improve the training effect, the ReduceLROnPlateau learning rate scheduler is used. When the loss on the validation set is not improved within 5 consecutive epochs, the learning rate will be reduced to 0.1 times the original until it drops to a minimum of 0.00001. In order to prevent overfitting, L2 regularization is used. The early stopping method is also applied during the training process, that is, the training is stopped when the accuracy of the validation set has not improved within 10 consecutive epochs. The batch size of the model is set to 64, the total number of training rounds is 2000, and the classification results are finally output.

[0078] The loss function of the model is the cross entropy loss function, which is expressed as

[0079]

[0080] in shows the cross entropy loss, y k The true category label of the kth training sample, represents the predicted value of the kth training sample, N bIndicates the total number of samples in a training batch. During training, to avoid overfitting of the model, the L2 regularization method is used to penalize the model weights, and the regularization coefficient is set to 0.0001.

[0081] At the same time, the Early Stopping method is used. Once the accuracy of the validation set does not improve within 10 consecutive epochs, the training is stopped.

[0082] The experimental dataset is the motor imagery dataset 2a of the 4th International Brain-Computer Interface Competition. This dataset contains EEG signal data of 9 subjects when they performed four different motor imagery tasks, which were to imagine the movements of the left hand, right hand, foot, and tongue. The experiment for each subject was divided into two stages, each stage contained 288 experiments, and each motor imagery task was performed an average of 72 times. In this study, we selected the data from the 2 to 6 second time period of each experiment as independent samples, which means that each sample is 4 seconds long, contains 1000 sampling points, and uses 22 channels of data. In the experiment, we used the data from the first stage to train the model, and the data from the second stage was used to test the model.

[0083] Figure 6 A bar chart showing the classification results for all subjects. Figure 7 The training loss and accuracy curves of four subjects A02, A04, A06, and A09 are shown.

[0084] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or replace some of the technical features therein by equivalents. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for recognizing electroencephalogram signals based on Kans-Transformer, characterized in that: include: Preprocessing module: obtain the original EEG signal data, use filters to filter, and at the same time, enhance the data through data enhancement technology to obtain enhanced EEG data; Time domain and spatial domain feature extraction module: Use deep convolutional network to extract features of EEG signals, including time domain and spatial domain features, and introduce ECA module to perform channel attention weighting after time domain convolution. At the same time, integrate time domain feature information of different scales through feature fusion technology; Kans-Transformer module: Kolmogorov-Arnold Networks (Kans) are introduced into the Transformer Encoder module to obtain the Kans-Transformer module, which further processes the fused features; Classification module: This module contains convolutional layers, normalization layers, activation function layers and linear layers. The classifier receives the features processed by the Kans-Transformer module and outputs the classification labels of the left hand, right hand, feet and tongue movement imagination. During the training process, the model parameters are adjusted through the back propagation algorithm, and the cross entropy loss function is used as the optimization objective function.

2. The method for recognizing electroencephalogram signals based on Kans-Transformer according to claim 1, characterized in that: The preprocessing module further comprises: The MI-EEG signal was filtered in two steps using a multi-order Butterworth bandpass filter algorithm and a notch filter; The signal segment reconstruction data enhancement technology is implemented on the filtered data.

3. The method for recognizing electroencephalogram signals based on Kans-Transformer according to claim 1, characterized in that: The temporal and spatial feature extraction modules include: A deep convolutional network is used to extract the time domain and spatial domain features of EEG signals, and the ECA module is introduced after the time domain convolution to perform channel attention weighting. At the same time, the time domain feature information of different scales is integrated through feature fusion technology.

4. The method for recognizing electroencephalogram signals based on Kans-Transformer according to claim 1, characterized in that: The Kans-Transformer module includes: The Multilayer Perceptron (MLP) in the Transformer Encoder module was replaced with Kolmogorov-Arnold Networks (Kans) to improve the way the model processes EEG signal features.

5. The method for recognizing electroencephalogram signals based on Kans-Transformer according to claim 1, characterized in that: The classification module comprises: Design a classifier that combines convolutional layers, normalization layers, activation function layers, and linear layers. The classifier receives spatiotemporal features from the improved Transformer Encoder module (Kans-Transformer unit) and classifies based on these features. During training, the model generates predicted labels and compares them with the true labels, measuring the difference by calculating the cross entropy loss function. Use the back-propagation algorithm to update the model parameters based on the loss value.

6. A Kans-Transformer-based EEG signal recognition system, characterized in that: include: Preprocessing unit: used to filter, normalize and perform data enhancement operations on the original EEG signals; Time domain and space domain feature extraction unit: used to extract the time domain and space domain features of EEG signals and perform feature fusion to obtain a more comprehensive feature representation; Kans-Transformer unit: Kolmogorov-Arnold Networks (Kans) are introduced into the Transformer Encoder module to improve the model's ability to capture EEG signal features; Classification model unit: contains convolutional layer, normalization layer, activation function layer and linear layer; at runtime, the classifier receives the features output by the Kans-Transformer unit and outputs the corresponding classification label; during training, the cross entropy loss function value is calculated by comparing the predicted label and the true label, and then the model parameters are adjusted according to this loss value through the back propagation algorithm.

Images