Cross-subject movement phenomenon electroencephalogram signal classification method based on TSLANet and Riemannian geometric features

Abstract

The invention discloses a cross-subject movement phenomenon electroencephalogram signal classification method based on TSLANet and Riemannian geometric features, and relates to the technical field of electroencephalogram signals. The classification method is a new motor imagery electroencephalogram classification method based on a deep learning model, the method comprises the steps that firstly, original electroencephalogram signals are subjected to filtering and normalization processing to serve as input of the novel deep learning model RMETNet, the RMETNet combines a convolutional network and a TSLANet network, complex time sequence features are extracted through the TSLANet, and then the complex time sequence features are extracted through the deep learning model RMETNet; meanwhile, Riemannian geometric features are learned through a multi-scale convolution module, and the RMETNet combines time sequence features with features after Riemannian geometric alignment, so that feature expression is enriched, the accuracy of motor imagery electroencephalogram classification is improved, in order to reduce cross-subject distribution feature differences, MMD loss is introduced in the training process of the RMETNet, and the accuracy of motor imagery electroencephalogram classification is improved. The distribution consistency between the source domain and the target domain is enhanced, so that the effect of the cross-subject experiment is improved, the superiority of the proposed model is evaluated by carrying out the single-subject experiment and the cross-subject experiment on the two common data sets, and a good result is obtained.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signals, and in particular to an EEG signal classification method for cross-subject motion phenomena based on TSLANet and Riemannian geometry features. Background Art

[0002] Brain-computer interface technology achieves direct interaction between the human brain and external devices by decoding electrophysiological signals generated by brain activity, bringing revolutionary breakthroughs to medical rehabilitation, intelligent control and other fields. Among them, motor imagery electroencephalogram (MI-EEG), as a non-invasive, high-temporal resolution neural signal, has become one of the core control signals of the BCI system. This is a process in which the subject triggers the neural activity of the motor cortex of the brain by mentally simulating specific actions (such as left hand, right hand or foot movements) without actual limb movement. This process will induce event-related desynchronization (ERD) and event-related synchronization (ERS) phenomena, which are manifested as significant changes in the energy of μ rhythm (8-12Hz) and β rhythm (13-30Hz). The changes in the energy of these frequency bands have become markers related to biological control and have become an important physiological basis for studying signal characteristics in this field.

[0003] In recent years, EEG-related research on motor imagery has flourished due to its great potential in clinical rehabilitation and intelligent interaction. However, the decoding of MI-EEG signals still faces multiple challenges: first, the signal has low signal-to-noise ratio, non-stationary and high-dimensional characteristics, and is easily interfered by electromyographic artifacts and environmental noise, which makes it very difficult to extract EEG features. Simple classical methods are easily limited by noise; second, there are significant physiological differences between individuals. The same action imagination may induce very different EEG responses in different subjects, resulting in a sharp decline in cross-subject classification performance. Specifically, the experimental parameters on a single subject are difficult to apply to other subjects. The parameter adjustment process for each individual greatly increases the experimental and application costs, which further limits the development and implementation of brain-computer interfaces; third, traditional methods rely on manual feature extraction (such as CSP, power spectrum analysis), which is difficult to capture complex spatiotemporal dynamic patterns. Although traditional methods can achieve certain results in simple EEG signal processing, with the enrichment and complexity of EEG data, when faced with more diverse EEG tasks, such as when there are more EEG classifications, traditional methods will show a cliff-like performance decline; in order to make a breakthrough from these problems, a novel research method is urgently needed.

[0004] Since Hinton proposed the concept of deep learning, this technology has achieved major breakthroughs in multiple disciplinary fields such as computer vision and natural language processing, promoting the rapid development of applications such as face recognition and autonomous driving. The core advantage of deep learning lies in its ability to automatically extract high-dimensional features from data without relying on prior knowledge designed manually, which is also different from traditional feature extraction methods. Traditional manual extraction techniques limit the generalization ability of the model because it is difficult to fully capture the spatio-temporal dynamic patterns of signals. At this time, applying deep learning technology can break through the performance bottleneck. Under the condition of sufficient data volume, the research on electroencephalogram (EEG) signals can be effectively combined with deep learning technology. Taking advantage of the characteristic of automatically extracting high-dimensional features, it can break through the limitations of traditional methods. Representative convolutional neural networks (CNNs) and temporal modeling methods can automatically extract multi-scale spatio-temporal features of MI-EEG through end-to-end learning, significantly improving the classification accuracy of EEG signals. Although the existing deep learning technology has far exceeded traditional methods in the ability of single-subject feature extraction and reached a considerable level, in order to further solve another major problem in EEG research: the cross-subject problem, the general training methods are no longer suitable. These methods cannot achieve feature mapping between different subjects. From this perspective, transfer learning strategies have been introduced into EEG analysis research.

[0005] Although some progress has been made in EEG signal analysis, there are still some challenges in dealing with spatio-temporal features by existing methods. Existing methods often focus on the strong time-frequency characteristics of signals and ignore the spatial distribution of signals and the interactions between brain regions. Although this method can extract dynamic changes in time, it cannot make full use of the spatial correlation information between EEG electrodes, resulting in limited accuracy and reliability of emotion recognition. In addition, the non-linearity and non-stationarity of EEG signals require more complex analysis methods to understand the complexity of brain activities. Existing methods often lack flexibility and adaptability and cannot effectively extract deep features from multiple dimensions.

[0006] Therefore, it is necessary to develop a new method that can not only handle the time dynamics of EEG signals but also take into account the complexity of spatial distribution to provide more comprehensive and accurate emotion recognition, which becomes the key research direction to promote the practical application of MI-EEG technology. Summary of the Invention

[0007] The purpose of the present invention is to provide a cross-subject motor imagery EEG signal classification method based on TSLANet and Riemannian geometric features to solve the technical problems raised in the background art.

[0008] To achieve the above purpose, the present invention provides the following technical solution: A cross-subject motor imagery EEG signal classification method based on TSLANet and Riemannian geometric features, which at least includes the following steps:

[0009] S1: Build the TSLANet module, which is used to effectively extract complex temporal features in the time-frequency space. The TSLANet module includes an adaptive spectrum module and an interactive convolution block, namely the ASB module and the ICB module. The ASB module and the ICB module form a single layer that can be extended to multiple layers;

[0010] S2: Build the time-frequency convolution module, which is used to enable the model to have the ability to learn local temporal features in subsequent model construction;

[0011] S3: Build the multi-scale Riemannian convolution module, which is used to process the Riemannian geometric features of EEG signals. By calculating the covariance matrix of the source domain and target domain data, it maps them to the Riemannian geometric space to obtain the feature representation after Riemannian geometric alignment

[0012] S4: Build the classifier module. Use two fully connected layers as the classifier model. After passing through the Softmax function, it outputs an M-dimensional vector. Use cross-entropy as the loss function:

[0013]

[0014] where M represents the number of EEG categories; y and are the true value and the predicted label respectively; N b represents the number of trials in a batch;

[0015] S5: Based on the TSLANet module, the time-frequency convolution module, the multi-scale Riemannian convolution module and the classifier module, build the RMETNet model for EEG signal classification. The RMETNet model is an end-to-end network module;

[0016] S6: Train the RMETNet model. Use the unsupervised distribution alignment method and use the trained RMETNet model for cross-subject motor imagery EEG signal classification.

[0017] Furthermore, the ASB module uses the Fourier transform to convert the EEG time series data into frequency domain data, applies an adaptive threshold to attenuate high-frequency noise and highlight the relevant spectral features in the EEG signal. After processing, it uses the inverse fast Fourier transform to reconstruct the time domain features, reducing noise while enhancing the time series representation;

[0018] The ICB module is a lightweight convolution block that uses different kernel sizes to refine features interactively, thereby improving the adaptability to dynamic changes in the time series, achieving a dynamic unity in the extraction of local and global time change features of time series analysis

[0019] Furthermore, the TSLANet module is composed of stacked TSLANets;

[0020] The input multi-channel EEG signal passes through the embedding layer to obtain deep features

[0021] where T is the time step; d model is the feature dimension; is the initial representation of the signal input into the TSLANet module;

[0022] For the TSLANet of the L-th layer, its input is Then, the deep temporal changes of the EEG signal are extracted:

[0023]

[0024] Specifically, TSLANet includes the following processes:

[0025] First, the spectral features of each time series channel are obtained through the fast Fourier transform, thereby generating a comprehensive frequency domain representation F, that is, the original EEG frequency domain data, for the input one-dimensional temporal features The frequency domain representation is obtained by performing the fast Fourier transform along the spatial dimension, and the calculation is as follows:

[0026]

[0027] where FTT[] represents the calculation of the one-dimensional fast Fourier transform; L` represents the length of the EEG signal frequency domain sequence; indicates that the frequency domain feature is in the form of a complex number; C represents the number of channels of the signal;

[0028] Each channel of the signal undergoes an independent transformation, generating a comprehensive frequency domain representation F, which represents the spectral features of the original EEG across all channels;

[0029] Secondly, after obtaining the high-frequency noise, the high-frequency components usually represent the fluctuations of the signal deviating from the potential trend. Here, an adaptive local filter is introduced, which can dynamically adjust the filtering level according to the data characteristics and remove the high-frequency noise components, which is important for processing non-stationary EEG signals because the non-stationary characteristics cause its spectrum to change rapidly over time, and the filter will adaptively set the frequency threshold;

[0030] The signal power spectrum of F is calculated, which helps to identify the components of the main frequencies in the EEG signal. The calculation of the power spectral density is the square of the amplitude of the frequency component: P = |F| 2 , providing a measure of the different frequency intensities in the EEG signal sequence;

[0031] The key to achieving effective noise reduction of EEG lies in adaptively filtering out high-frequency components from the power spectrum P, which is achieved through a trainable threshold θ. The threshold θ is adjusted according to the spectral characteristics of the data. The threshold θ is set as a learnable parameter so that it can be optimized by backpropagation during the training process. Specifically, represents the loss function The gradient of θ enables θ to identify the basic signal frequencies and noise. The adaptive threshold formula is as follows:

[0032] F filtered = F ⊙ (P > θ)

[0033] where ⊙ represents element-wise multiplication; (P > θ) is a binary mask, and the frequencies higher than the threshold θ are retained. This step ensures the effective removal of high-frequency noise while retaining key information. By adaptively selecting the threshold, the adaptive spectrum module can customize the filtering process according to each EEG signal data, improving the effectiveness of the model in processing EEG data;

[0034] After adaptively filtering the high-frequency data noise, two sets of learnable filters are adopted. A global filter learns from the frequency-domain data F of the original EEG, and a local filter learns from the adaptively filtered EEG frequency-domain data F filtered ;

[0035] Let W G and W L represent the learnable global filter and local filter respectively. The formulas are as follows:

[0036] F G = W G ⊙ F

[0037] F L = W L ⊙ F filtered ,

[0038] Then, the filtered features are integrated to capture all the features, i.e., F intergrated = F G + F L ;

[0039] After obtaining all the features, the EEG frequency-domain features are transformed back to the time domain by applying the inverse fast Fourier transform to obtain the time-domain signal X 1D `, and the formula calculation is as follows:

[0040]

[0041] The inverse fast Fourier transform ensures that the enhanced EEG features are consistent with the structure of the original input EEG signal data;

[0042] After enhancing the feature representation through the adaptive frequency module, the interactive convolution module is used to capture local features and long-range dependencies. Specifically, the content is as follows:

[0043] In the first convolutional layer, a small convolutional kernel size is tried to capture fine-grained local features in the EEG data. In the second layer, a larger convolutional kernel is used to identify broader and longer-range dependencies. Given the output X of the inverse fast Fourier transform, 1D `, which is used as the input to the interactive convolution module. The process is as follows:

[0044] A1 = φ(Conv1(X 1D `) ⊙ Conv2(X 1D `)

[0045] A2 = φ(Conv2(X 1D `) ⊙ Conv1(X 1D `)

[0046] where Conv1(*) and Conv2(*) are two one-dimensional convolutional operations; φ is the GELU activation function; then through the final convolutional layer Conv3(*):

[0047] X 1D = Conv3(A1 + A2)

[0048] where the output X 1D represents the enhanced EEG signal features prepared for the last layer of the TSLANet module. Then, the features of the TSLANet module are input into the next time-frequency convolutional module.

[0049] Furthermore, the time-frequency convolutional module uses two cascaded convolutional layers and adopts a two-dimensional convolutional kernel (1, K e ) to capture time information, denoted by ;

[0050] adopts a two-dimensional convolutional kernel (C`, 1) to capture spatial information, denoted by , where the size of C` is related to the channels of the EEG signal. By learning the spatial filter through spatial convolution, that is, related to the EEG channels, such a structure can effectively extract the local information of the EEG signal in the time dimension and the global features in the spatial dimension;

[0051] Let X 1D be the 1D feature sample output by the TSLANet module. The output sample is expanded into a 2D sample using the following formula:

[0052] X 2D = ExpendDim(X 1D , 1)

[0053] Among them, ExpendDim(*, 1) represents the operation of expanding the dimension in the second dimension, that is, from become Through this operation, the features can be matched with the two-dimensional convolutional kernel; in order to maintain the non-stationary characteristics of the signal, the square activation function is used for processing; in order to reduce noise and the number of model parameters, the pooling layer is used to downsample the feature map; in order to make the average value of the latent features zero, improve the training stability, and speed up the training process, batch normalization is performed after the activation function;

[0054] Use X 2D to represent the output of the convolution operation, where b is the number of samples, f is the number of channels of the output feature map of the convolution module, c is the number of channels, and s is the number of sample features, which are defined as follows:

[0055]

[0056] Among them, and represent the sizes of two convolutional kernels; X 2D is the input feature; Conv2D() represents the two-dimensional convolution operation using two convolutional kernels; Square() represents the square activation function; AvgPool() represents the average pooling operation;

[0057] In order to ensure the stability of the feature distribution and avoid the problem of internal covariate shift in the model, after the square activation function, a logarithmic activation function and a batch normalization operation are added, which are defined as follows:

[0058] X 2D = BatchNorm(Log(X′ 2D ))

[0059] Among them, X′ 2D is the output feature after three convolutions and average pooling operations, Log() represents the logarithmic activation function, and BatchNorm() represents the batch normalization operation.

[0060] Furthermore, the original EEG signal input by the multi-scale Riemannian convolution module is First, the covariance matrix is calculated, and the formula is as follows:

[0061]

[0062] Among them, Cov represents the covariance matrix between channels, that is, the covariance matrix calculated from the source domain signal data and the target domain signal data. Then, the Riemannian geometric distance between the two matrices is calculated using the Riemannian geometry distance, which is defined as follows:

[0063]

[0064] Among them, log(Cov1) represents the matrix logarithm of the covariance matrix. Through eigenvalue calculation, all covariance matrices are mapped to a reference point in Riemannian geometry (usually the average of the overall covariance matrix), and the geometric mean is calculated as Translate each covariance matrix Cov i to the reference point The calculation formula is as follows:

[0065]

[0066] For the aligned covariance matrices, a specific structural design is carried out, multi-scale features are extracted and the robustness of data points is enhanced. Three convolutional layers with different kernel sizes are used to implement feature extraction at three levels: local range, medium scale, and global distribution models, and then feature fusion is performed;

[0067] Finally, it is flattened into a low-dimensional feature space to generate a compact feature representation:

[0068]

[0069] X rout =Linear(Flatten(X cat ))

[0070] Then, the features output by the multi-scale Riemannian module are fused with the time-frequency convolution module, and then the model is sent to the classifier module.

[0071] Furthermore, the input of the RMETNet model is the preprocessed motor imagery EEG signal data, and the RMETNet model includes two branches;

[0072] The first branch is input with the original EEG data, and the TSLANet module is used to capture the time feature pattern of the EEG signal. Then, the time series features with a higher-level representation are output through the time convolution and depth convolution in the time-frequency convolution module;

[0073] The second branch takes the Riemannian features calculated from the source domain and the target domain as input. The second branch extracts comprehensive Riemannian geometric features through the multi-scale Riemannian convolution module as different cross-subject distribution features;

[0074] Finally, the features of the two branches are combined and sent to a fully connected layer (FC) with a SoftMax classifier, that is, the classifier module, so as to obtain the probability result of the motor imagery EEG signal classification.

[0075] Furthermore, the unsupervised distribution alignment method uses the MMD loss to calculate the feature distribution difference as the domain alignment loss, quantifies the difference between the source domain and the target domain, can reduce the distribution difference between the source domain and the target domain, align their global distributions, and the features across different subject domains are realized by sharing network parameters. The process is defined as follows:

[0076]

[0077] Among them represents the classification loss; λ represents the pair, calculated from the source domain labels; represents the MMD loss, which realizes minimizing the distribution difference between the feature classifications of the source domain and the target domain. Its calculation formula is as follows:

[0078]

[0079] where F s , F t represents the feature representations obtained by inputting the source domain data X s and the target domain data X t into the network.

[0080] Compared with the prior art, the beneficial effects of the present invention are:

[0081] The classification method proposed by the present invention is a new method for motor imagery EEG classification based on a deep learning model. This method first filters and normalizes the original EEG signals as the input of the new deep learning model RMETNet. RMETNet combines a convolutional network and a TSLANet network, uses TSLANet to extract complex temporal features, and at the same time learns Riemannian geometric features through a multi-scale convolutional module. RMETNet combines the temporal features with the features after aligning the Riemannian geometry, enriching the feature expression, thereby improving the accuracy of motor imagery EEG classification. In order to reduce the cross-subject distribution feature differences, RMETNet introduces the MMD loss during the training process, enhancing the distribution consistency between the source domain and the target domain, thereby improving the effect of cross-subject experiments. By conducting single-subject and cross-subject experiments on two public datasets, the superiority of the proposed model is evaluated and good results are obtained. Description of the Drawings

[0082] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0083] Figure 1 is the overall structure of RMETNet of the present invention;

[0084] Figure 2 This is the complete training schematic diagram of the RMETNet of the present invention;

[0085] Figure 3 This is the structural diagram of the time-frequency convolution module of the present invention;

[0086] Figure 4 This is the visualization schematic diagram of the test data characteristics of subject S1 in BCICIV2a of the present invention under two conditions of whether to apply the MMD loss method and Riemann features. Detailed implementation manners

[0087] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments.

[0088] Before specifically describing the implementation process of the present invention, it is necessary to understand that the proposed TSLANet provides a new direction for the processing of time series signals. Two key lightweight adaptive spectrum modules are used to replace self-attention. This module covers the entire spectrum and captures the key long-term and short-term interactions of the data, similar to circular convolution. Secondly, the high frequency is selectively attenuated by an adaptive threshold method. Finally, interactive convolution is used, which is controlled by a convolutional neural network with different kernel sizes to enrich the model's ability to capture and interpret complex time patterns. Based on this time series model, a new method for classifying motor imagery electroencephalogram signals based on a deep learning model is proposed. First, the original electroencephalogram data is filtered and normalized. Secondly, the Riemannian geometric alignment of the source domain training set and the target domain test set is calculated. Finally, the EEG-TSLANet model is used for motor imagery classification. This model uses TSLANet to obtain time series information, and a convolutional neural network module is used to extract dynamic spatio-temporal features, which are fused with Riemannian geometric information.

[0089] A method for classifying cross-subject motor imagery electroencephalogram signals based on TSLANet and Riemannian geometric features is classified by the RMETNet model.

[0090] Such as Figure 1The architecture of RMETNet is shown, and the structures of each layer of the network are described in detail in Table 1. The proposed end-to-end network consists of three main blocks: the TSLANet module, the time-frequency convolution module, the multi-scale Riemannian convolution module, and the classification module. The input of the model is the preprocessed motor imagery EEG signal data, which includes the original EEG input and the Riemannian features calculated from the source domain and the target domain. The model has two branches. The first branch takes the original EEG data as input, captures the temporal feature patterns of the EEG signal through TSLANet, and then outputs the time series features with higher-level representation (Higher Level Representation) through the temporal convolution and depth convolution in the time-frequency convolution module. The second branch takes the Riemannian features as input, and this module can extract comprehensive Riemannian geometric features as different cross-subject distribution features. Finally, the features of the two branches are combined and fed into the fully connected layer (FC) with a SoftMax classifier to obtain the probability results of the classification of motor imagery EEG signals.

[0091] Table 1 Detailed parameters of the RMETNet structure

[0092]

[0093]

[0094] Among them: C represents the number of EEG channels, T represents the number of time samples, F1 represents the number of temporal filters, F2 represents the number of spatial filters, K E represents the size of the first convolutional kernel, K A represents the size of the one-dimensional convolutional kernel in the ECA module, F T represents the number of filters in the ECA module. The size of the Dropout layer in the framework is.

[0095] The RMETNet module is a transfer learning framework that endows the neural network with the ability to transfer features from the source domain to the target domain. The network mainly includes the TSLANet module, which is used to learn the temporal features of EEG signals, the time-frequency convolution module, which extracts the time-frequency domain features of EEG, and the multi-scale convolution module, which focuses on cross-subject features for cross-subject feature alignment. As Figure 2 shown, the model features use the MMD loss to calculate the feature distribution difference as the domain alignment loss. This is an unsupervised distribution alignment method that quantifies the difference between the source domain and the target domain, can reduce the distribution difference between the source domain and the target domain, align their global distributions, and the cross-subject inter-domain features are realized by sharing network parameters. The process is defined as follows:

[0096]

[0097] Among them represents the classification loss, and λ represents the pair, which is calculated from the source domain labels. represents the MMD loss, which minimizes the distribution difference between the feature classifications of the source domain and the target domain. Its calculation formula is as follows:

[0098]

[0099] where F s , F t represent the feature representations obtained by inputting the source domain data X s and the target domain data X t into the network.

[0100] The specific steps are as follows:

[0101] S1: Build the TSLANet module, which is used to effectively extract complex time features in the time-frequency space. The TSLANet module includes an adaptive spectrum module and an interactive convolution block, namely the ASB module and the ICB module. The ASB module and the ICB module form a single layer that can be extended to multiple layers;

[0102] In the present invention, a time series model structure is used to extract time pattern features simultaneously. Different from other sequential types, the characteristic of a time series is continuous recording and the records only save some scalars, which results in that the data at a single time point cannot provide enough information for analysis. For example, for the time-frequency information of electroencephalogram (EEG), although the time-frequency metrics (such as time-frequency power, phase synchronization information, etc.) are very useful for signal feature analysis of EEG signals with prominent signal-to-noise ratio. In such a case, we cannot achieve high-precision feature extraction, and thus it is very easy to lose the rich time information embedded in the EEG signal. However, the TSLANet model can extract spectral features and use hybrid convolution operations to extract cross time-frequency information. To sum up, this module can effectively extract complex time features in the time-frequency space.

[0103] The ASB module uses the Fourier transform to convert the EEG time series data into frequency domain data, applies an adaptive threshold to attenuate high-frequency noise and highlight the relevant spectral features in the EEG signal, and then uses the inverse fast Fourier transform to reconstruct the time domain features after processing, reducing the noise while enhancing the time series representation;

[0104] The ICB module is a lightweight convolution block that uses different kernel sizes to refine features interactively, thereby improving the adaptability to dynamic changes in the time series, and achieving a dynamic unity in the extraction of local and global time change features of time series analysis

[0105] The TSLANet module is composed of stacked TSLANets;

[0106] The input multi-channel EEG signals are passed through an embedding layer to obtain deep features

[0107] where T is the time step; d model is the feature dimension; is the initial representation of the signal input into the TSLANet module;

[0108] For the TSLANet of the L-th layer, its input is Then, the deep temporal changes of the EEG signals are extracted:

[0109]

[0110] Specifically, TSLANet includes the following processes:

[0111] First, the spectral features of each time series channel are obtained through the fast Fourier transform, thereby generating a comprehensive frequency domain representation F, that is, the original EEG frequency domain data, for the input one-dimensional temporal features The frequency domain representation is obtained by performing the fast Fourier transform along the spatial dimension, and the calculation is as follows:

[0112]

[0113] where FTT[] represents the calculation of the one-dimensional fast Fourier transform; L` represents the length of the EEG signal frequency domain sequence; indicates that the frequency domain features are in the form of complex numbers; C represents the number of channels of the signal;

[0114] Each channel of the signal undergoes an independent transformation, generating a comprehensive frequency domain representation F, which represents the spectral features of the original EEG across all channels;

[0115] Secondly, after obtaining the high-frequency noise, the high-frequency components usually represent the fluctuations of the signal deviating from the potential trend. Here, an adaptive local filter is introduced, which can dynamically adjust the filtering level according to the data characteristics and remove the high-frequency noise components. It plays an important role in processing non-stationary EEG signals because the non-stationary characteristics cause its spectrum to change rapidly over time, and the filter will adaptively set the frequency threshold;

[0116] The signal power spectrum of F is calculated, which helps to identify the components of the main frequencies in the EEG signals. The calculation of the power spectral density is the square of the amplitude of the frequency component: P = |F| 2 , providing a measure of the different frequency intensities in the EEG signal sequence;

[0117] The key to achieving effective noise reduction of EEG lies in adaptively filtering out high-frequency components from the power spectrum P, which is achieved through a trainable threshold θ. The threshold θ is adjusted according to the spectral characteristics of the data and is set as a learnable parameter so that it can be optimized by backpropagation during training. Specifically, represents the loss function The gradient of θ enables θ to identify the basic signal frequencies and noise. The formula for this adaptive threshold is as follows:

[0118] F filtered = F ⊙ (P > θ)

[0119] where ⊙ represents element-wise multiplication; (P > θ) is a binary mask, and frequencies higher than the threshold θ are retained. This step ensures that high-frequency noise is effectively removed while key information is retained. By adaptively selecting the threshold, the adaptive spectrum module can customize the filtering process according to each EEG signal data, improving the effectiveness of the model in processing EEG data;

[0120] After adaptively filtering high-frequency data noise, two sets of learnable filters are adopted. One global filter learns from the frequency-domain data F of the original EEG, and one local filter learns from the adaptively filtered EEG frequency-domain data F filtered ;

[0121] Let W G and W L represent the learnable global filter and local filter respectively, and the formulas are as follows:

[0122] F G = W G ⊙ F

[0123] F L = W L ⊙ F filtered ,

[0124] Then, the filtered features are integrated to capture all features, that is, F intergrated = F G + F L ;

[0125] After obtaining all features, the EEG frequency-domain features are transformed back to the time domain by applying the inverse fast Fourier transform to obtain the time-domain signal X 1D `, and the formula calculation is as follows:

[0126]

[0127] The inverse fast Fourier transform ensures that the enhanced EEG features are consistent with the structure of the original input EEG signal data;

[0128] After enhancing the feature representation through the adaptive frequency module, the interactive convolution module is used to capture local features and long-range dependencies. Specifically, the content is as follows:

[0129] In the first convolutional layer, a small convolutional kernel size is attempted to capture fine-grained local features in the EEG data. In the second layer, a larger convolutional kernel is used to identify broader and longer-range dependencies. Given the output X of the inverse fast Fourier transform, 1D `, which is used as the input to the interactive convolution module. The process is as follows:

[0130] A1 = φ(Conv1(X 1D `)) ⊙ Conv2(X 1D `)

[0131] A2 = φ(Conv2(X 1D `)) ⊙ Conv1(X 1D `)

[0132] where Conv1(*) and Conv2(*) are two one-dimensional convolutional operations; φ is the GELU activation function; then through the final convolutional layer Conv3(*):

[0133] X 1D = Conv3(A1 + A2)

[0134] where the output X 1D represents the enhanced EEG signal features prepared for the last layer of the TSLANet module, and then the features of the TSLANet module are input into the subsequent time-frequency convolutional module.

[0135] S2: Build a time-frequency convolutional module, which is used to enable the model to have the ability to learn local time features in the subsequent model building; the time-frequency convolutional module is as Figure 2 shown

[0136] The time-frequency convolutional module uses two cascaded convolutional layers and adopts a two-dimensional convolutional kernel (1, K e ) to capture time information, denoted by ;

[0137] Adopt a two-dimensional convolutional kernel (C`, 1) to capture spatial information, denoted by where the size of C` is related to the channels of the EEG signal. By learning spatial filters through spatial convolution, that is, related to the EEG channels, such a structure can effectively extract the local information of the EEG signal in the time dimension and the global features in the spatial dimension;

[0138] Let X 1D be the 1D feature sample output by the TSLANet module, and use the following formula to expand the output sample into a 2D sample:

[0139] X 2D = ExpendDim(X 1D , 1)

[0140] where ExpendDim(*, 1) represents the operation of expanding the dimension in the second dimension, that is, from to Through this operation, the features can match the two-dimensional convolution kernel; in order to maintain the non-stationary characteristics of the signal, a square activation function is used for processing; in order to reduce noise and the number of model parameters, a pooling layer is used to downsample the feature map; in order to make the average value of the latent features zero, improve the training stability, and speed up the training process, batch normalization is performed after the activation function;

[0141] Let X 2D represent the output of the convolution operation, where b is the number of samples, f is the number of channels of the output feature map of the convolution module, c is the number of channels, and s is the number of sample features, which are defined as follows:

[0142]

[0143] where, and represent the sizes of two convolution kernels; X 2D is the input feature; Conv2D() represents the two-dimensional convolution operation using two convolution kernels; Square() represents the square activation function; AvgPool() represents the average pooling operation;

[0144] In order to ensure the stability of the feature distribution and avoid the problem of internal covariate shift in the model, after the square activation function, a logarithmic activation function and batch normalization operations are added, which are defined as follows:

[0145] X 2D = BatchNorm(Log(X' 2D ))

[0146] where X' 2D is the output feature after three convolutions and average pooling operations, Log() represents the logarithmic activation function, and BatchNorm() represents the batch normalization operation.

[0147] S3: Build a multi-scale Riemannian convolution module. The multi-scale Riemannian convolution module is used to process the Riemannian geometric features of EEG signals. By calculating the covariance matrices of the source domain and target domain data, mapping them to the Riemannian geometric space, and obtaining the feature representation after Riemannian geometric alignment. Using the multi-scale Riemannian convolution module can not only capture the global and local structural features of EEG signals, but also effectively integrate the relationships between multi-channel signals. More importantly, the mapped data distributions are closer in the Riemannian space, thereby reducing the distribution differences between different subjects or different domains, achieving the purpose of cross-subject learning, and significantly improving the generalization ability of subsequent models;

[0148] The original EEG signal input to the multi-scale Riemannian convolution module is First, calculate the covariance matrix, and the formula is as follows:

[0149]

[0150] Among them, Cov represents the covariance matrix between channels, that is, the covariance matrix calculated from the source domain signal data and the target domain signal data. Then, use the Riemannian geometric distance to calculate the Riemannian geometric distance between the two matrices, and the definition is as follows:

[0151]

[0152] Among them, log(Cov1) represents the matrix logarithm of the covariance matrix. Through eigenvalue calculation, all covariance matrices are mapped to a reference point in Riemannian geometry (usually the average value of the overall covariance matrix), and the calculated geometric mean is Translate each covariance matrix Cov i to the reference point The calculation formula is as follows:

[0153]

[0154] Perform a specific structural design on the aligned covariance matrix, extract multi-scale features and improve the robustness of data points. Use three convolutional layers with different kernel sizes to implement feature extraction at three levels: local range, medium scale, and global distribution models, and then perform feature fusion;

[0155] Finally, flatten it into a low-dimensional feature space to generate a compact feature representation:

[0156]

[0157] X rout = Linear(Flatten(X cat ))

[0158] Next, fuse the features output by the multi-scale Riemann module with the time-frequency convolution module, and then send the model to the classifier module;

[0159] It should be noted that since the size of the Riemannian geometric features is determined based on the number of EEG signal channels, the Riemannian feature extraction modules obtained on the two datasets are different. That is, on the BCICIV2b dataset, there are only two convolutional layers with sizes 1×1 and 2×2.

[0160] S4: Build a classifier module, use two fully connected layers as the classifier model, output an M-dimensional vector after passing through the Softmax function, and use cross-entropy as the loss function:

[0161]

[0162] Among them, M represents the number of EEG categories; y and are the true value and the predicted label respectively; N b represents the number of trials in the batch;

[0163] S5: Build the RMETNet model for EEG signal classification based on the TSLANet module, the time-frequency convolution module, the multi-scale Riemann convolution module and the classifier module. The RMETNet model is an end-to-end network module;

[0164] The input of the RMETNet model is the preprocessed motor imagery EEG signal data. The RMETNet model includes two branches;

[0165] The first branch takes the original EEG data as input, captures the temporal feature pattern of the EEG signal through the TSLANet module, and then outputs the time series features with a higher-level representation through the temporal convolution and depth convolution in the time-frequency convolution module;

[0166] The second branch takes the Riemannian features calculated from the source domain and the target domain as input. The second branch extracts comprehensive Riemannian geometric features through the multi-scale Riemann convolution module as different cross-subject distribution features;

[0167] Finally, combine the features of the two branches and send them to the fully connected layer (FC) with a SoftMax classifier, that is, the classifier module, so as to obtain the probability result of the motor imagery EEG signal classification.

[0168] S6: Train the RMETNet model, use the unsupervised distribution alignment method, and use the trained RMETNet model for cross-subject motor imagery EEG signal classification.

[0169] The unsupervised distribution alignment method uses the MMD loss to calculate the feature distribution difference as the domain alignment loss, quantifies the difference between the source domain and the target domain, can reduce the distribution difference between the source domain and the target domain, align their global distributions, and the features across different subject domains are realized by sharing network parameters. The process is defined as follows:

[0170]

[0171] Among them represents the classification loss; λ represents the pair, calculated from the source domain labels; represents the MMD loss, which realizes minimizing the distribution difference between the feature classifications of the source domain and the target domain. Its calculation formula is as follows:

[0172]

[0173] where F s ,F t represent the feature representations obtained by inputting the source domain data X s and the target domain data X t into the network.

[0174] In summary:

[0175] The present invention proposes a new end-to-end deep learning network - RMETNet. This network combines a convolutional neural network (CNN), a TSLANet module, and a multi-scale Riemannian feature module, and also uses a domain adaptation training method based on MMD Loss, aiming to achieve cross-subject feature learning for motor imagery EEG signal classification. The design of this network has significant advantages: it does not require pre-training or complex preprocessing steps, and is applicable to both single-subject and cross-subject scenarios. According to the classification experiment results, RMETNet is superior to existing mainstream methods in the motor imagery EEG classification task, demonstrating its excellent performance in feature extraction and cross-subject domain adaptability.

[0176] First, the TSLANet module is effectively integrated into the network design, which is outstanding in capturing temporal and spatial features. Second, by introducing multi-scale Riemannian geometric alignment features in parallel, the network's ability to model cross-subject distribution features is further integrated, effectively alleviating the distribution shift problem caused by physiological differences between subjects to some extent. In addition, the network combines spatio-temporal convolutional layers to jointly learn local temporal and spatial features, improving the depth and breadth of feature extraction. In the cross-subject experiment, the maximum mean discrepancy loss (MMD Loss) is used as an auxiliary loss, and through this unique training mode, the distribution difference between different subjects is effectively reduced, enabling the network to demonstrate excellent robustness and generalization ability in cross-subject learning tasks. Compared with traditional CNN models, the proposed framework can learn more distinguishable and discriminative feature representations and achieve excellent classification results in cross-subject experiments.

[0177] To prove the performance of the TSLANet module adopted in the present invention, the following experiments are proposed:

[0178] Comparative experiment

[0179] To verify the performance advantages of the RMETNet algorithm proposed in the present invention in the task of motor imagery (MI) EEG signal feature recognition, subject-independent experiments and additional subject-dependent experiments are carried out. In addition, to further evaluate the effectiveness of the algorithm, the present invention also selects some representative excellent algorithms as benchmark models and conducts systematic comparative experiments with the method of the present invention on two public datasets. The specific descriptions of these benchmark algorithms are as follows:

[0180] FBCSP: A traditional machine learning method, the principle of which is based on the CSP-based spatial feature extraction algorithm to slice the frequency band and add a feature selection algorithm. Specifically, first slice the frequency band, then perform CSP filtering on each sub-band, and finally select features from the filtered features and output the classification result.

[0181] EEGNet: A compact convolutional neural network that learns temporal filters through convolutional kernels to capture motion-related frequency information. At the same time, the network adopts a separable convolution structure, including depthwise convolution and pointwise convolution, for learning spatial filters. This design significantly reduces the number of training parameters of the model while retaining good feature extraction ability.

[0182] ShallowConvNet: The data transformation process performed by ShallowConvNet is similar to the feature extraction method of FBCSP. Its core lies in the joint modeling of temporal and spatial features. Specifically, ShallowConvNet extracts spatio-temporal features through temporal convolution and spatial filters, and then applies a square non-linear transformation, a mean pooling layer, and a logarithmic activation function to further extract deep features and enhance their discriminability.

[0183] EEG-TCNet: EEG-TCNet is proposed as an extended model of EEGNet. After EEGNet completes the initial feature extraction, it introduces a Temporal Convolutional Network (TCN) structure to further mine the feature information in the time dimension. This design combines the efficient feature extraction ability of EEGNet with the modeling ability of TCN for long-term dependencies, thus enhancing the expression ability and classification performance of the temporal characteristics of EEG signals.

[0184] EEG-ITNet: This model introduces the Inception module and causal convolution with dilation, which can efficiently extract rich spectral, spatial, and temporal features from multi-channel EEG signals. The Inception module captures features in different frequency bands through multi-scale convolution, while the dilated causal convolution models long-term dependencies using an extended receptive field, thus comprehensively enhancing the feature expression ability and classification performance of EEG signals.

[0185] DS-TKL: This method realizes cross-domain learning by selecting discriminative features from the source domain and performing pseudo-label correction in the target domain. The DS-KTL method first preprocesses the samples through centroid alignment to reduce the distribution difference between the source domain and the target domain. Subsequently, Riemannian tangent space features are used for feature adaptation to further promote the alignment of the feature spaces between the source domain and the target domain. During the feature adaptation process, double selection is implemented by introducing a regularization mechanism to optimize the feature selection process and significantly improve the classification performance during the iteration process.

[0186] Transformer-Based Method: This method proposes an end-to-end EEG signal decoding algorithm based on swarm intelligence theory and virtual adversarial training. This algorithm combines the Transformer mechanism to optimize traditional convolutional classification methods. By introducing the self-attention mechanism, the research aims to extend the receptive field of EEG signals to global dependencies, thereby capturing more extensive spatio-temporal features. This method trains the neural network by optimizing the global parameters in the model, enhancing the global feature learning ability of the model while improving the classification performance of EEG signals.

[0187] EEGConformer: This is a compact Transformer structure based on EEG. The principle is to use a convolutional module to learn local features in one-dimensional time and space. The advantage of this model is that it then adopts a self-attention module to extract global dependencies within local time features. Compared with past global temporal feature extraction strategies, the model that integrates the self-attention mechanism learns more comprehensive features and achieves higher model performance.

[0188] We use multiple metrics to quantify the proposed model and compare it with benchmark classification methods simultaneously.

[0189] Results on BCIC IV 2a

[0190] Table 2 Comparison between methods across subjects on the BCIC IV 2a dataset

[0191] Methods S01 S02 S03 S04 S05 S06 S07 S08 S09 Avg.(kappa) EEGNet 65.53 54.06 73.42 53.10 61.04 49.69 66.87 63.12 74.24 50.60(0.27) ShallowConvNet 70.51 46.25 71.22 50.65 62.44 49.07 64.72 67.30 62.53 60.45(0.35) EEG-TCNet 67.40 62.94 79.90 49.44 60.00 51.60 71.52 71.11 69.12 62.35(0.45) EEG-ITNet 68.13 56.30 75.87 55.21 65.40 56.89 74.30 67.66 67.00 63.65(0.46) DS-TKL 65.97 45.83 81.94 69.44 64.58 48.61 78.47 95.14 81.25 70.14(～) Transformer-Based 61.61 59.12 64.35 62.36 67.35 63.02 65.62 63.67 64.97 63.56(0.46) EEG-TSLRNet 76.01 49.79 84.58 64.93 67.78 61.18 80.00 80.73 77.54 71.39(0.59)

[0192] *Among them, the binary classification experimental results of the DS-TKL using this dataset

[0193] Table 3 Comparison between methods of individual subjects on the BCIC IV 2a dataset

[0194]

[0195]

[0196] Table 2 shows the classification accuracy and Kappa scores of several advanced deep learning MI-EEG classification methods on individual subjects. These algorithms adopt a subject-dependent training method on the dataset. Table 3 shows the comparison of the subject-independent experimental results with other algorithms.

[0197] The proposed model RMETNet achieved a classification accuracy of 71.39% in cross-subject experiments, and this result used fixed parameters and hyperparameters for all test individuals. It can be clearly seen that the accuracy of our structure was significantly improved by 21.4% compared with the classical EEGNet algorithm. From the results, the performance of classical algorithm methods such as ShallowConvNet and EEGNet only reached an acceptable result in cross-subject results, showing great limitations. This indicates that although deep learning methods have strong feature representation capabilities, the use of only convolutional neural networks has very limited perception, which will affect the decoding accuracy of EEG signals, resulting in poor single-subject experimental results. At the same time, these deep learning methods do not pay attention to cross-subject features. For the differences between cross-subject data, they can only perform well in the source domain, but cannot directly consider the distribution differences between the target domain and the target domain. This makes their cross-subject results not show outstanding effects, which also proves that our cross-subject method can align the distribution in the cross-subject study of motor imagery EEG, optimize the features of the source domain and the target domain to be consistent, thereby improving the adaptive learning ability for target domain data.

[0198] In addition to cross-subject research, the present invention also compared the feature extraction capabilities of the single-subject, i.e., subject-related experiments to verify the model. After removing the MMD Loss and Riemannian geometric feature fusion, comparing only the proposed basic network with other models also showed consistent advantages. Specifically, the single-subject experiment on the BCICIV2a dataset reached 80.7%, which was about 1.5% higher than the accuracy of other deep learning methods. In the comparison on BCICIV2b

[0199] Cross-subject comparison result 3

[0200] Table 4 Comparison of cross-subject experimental methods on the BCICIV2b dataset

[0201]

[0202] Single-subject comparison result 4

[0203] Table 5 Comparison of single-subject experimental methods on the BCICIV2b dataset

[0204]

[0205] BCICIV2b is a dataset with low complexity for three-channel binary classification. The proposed model also achieved the best classification results in the cross-subject experiments on this dataset. It can be seen that all methods achieved higher accuracy rates on the BCICIV2b dataset, indicating that deep learning methods have good feature extraction capabilities in dealing with three-channel datasets with low complexity. In terms of predicting volatility, the average accuracy of RMETNet exceeded 86%. In the case of fewer channels, the network parameters of RMETNet were optimized for the BCICIV2b dataset in the experiment, adjusting the network channel parameters and using only two branches (only using 1*1 and 2*2 convolutional kernels) in the multi-scale Riemannian geometric feature extraction module. The classification performance in BCICIV2b reflects the good robustness of the proposed model.

[0206] Ablation experiment

[0207] Table 6 Ablation experiments on the BCICIV2a dataset

[0208]

[0209] Table 7 Ablation experiments on BCICIV2b

[0210]

[0211] The present invention systematically evaluated the necessity and contribution of the Riemannian feature module and the domain adaptation learning method based on MMDLoss to the improvement of model performance through ablation experiments. The proposed EEG-TSLANet model was used as the baseline in the experiments, and the performance of separately introducing the Riemannian geometric feature module, separately applying the MMDLoss loss function, and integrating both was verified respectively. The experimental results are shown in Table 6 and Table 7. The improvement of the overall model performance can be attributed to the contribution of each module. Specifically, Table 6 shows that on the BCICIV2a dataset, after integrating the Riemannian feature fusion and the domain adaptation method based on MMDLoss, the average performance improved by about 4.4%; separately applying the MMDLoss method could increase the average performance by XX%; separately applying the Riemannian feature fusion improved the average by 0.16%. The results in Table 7 show that on the BCICIV2b dataset, separately introducing the Riemannian feature fusion layer increased the average performance by about 0.23%; after separately applying the MMDLoss loss, the average accuracy increased by 1.42%; when the two methods were applied jointly, the overall average accuracy increased by 0.89%. These experimental results indicate that both the Riemannian feature module and the domain adaptation method based on MMDLoss can significantly improve the cross-subject performance of the model, and the best results can be obtained by applying both methods simultaneously.

[0212] Visualization

[0213] We use the UMAP visualization method to preserve the deep feature distribution of EEG signals, so as to explain the impact of the features extracted by the two methods cited in the model on the classification performance. We use a statistical dimensionality reduction and visualization learning technique, UMAP. In terms of visualization quality, the UMAP algorithm preserves more global structures and features are more sufficient compared to the t-SNE algorithm. After sufficient training, we select the feature distribution of Subject 1 in the BCICIV2a dataset as shown in the figure. For the test data, it can be clearly seen that without the MMDLoss method and Riemannian feature fusion, the distances between different categories are close, and the same phenomenon also occurs when the Riemannian feature fusion module is removed, that is, the intra-class distance of the extracted features is larger and the inter-class distance is smaller (see Figure 4 ).

[0214] In the experimental evaluation, RMETNet has achieved the current state-of-the-art performance on two public datasets, fully verifying its wide applicability in the motor imagery EEG classification task. Through ablation experiment analysis, we further prove the key role of the multi-scale Riemannian feature module and MMDLoss in improving the overall model performance. Especially in enhancing the cross-subject distribution alignment ability, the multi-scale Riemannian feature module plays an important role, while the introduction of MMDLoss significantly improves the cross-domain adaptation ability of the model. In addition, to prevent overfitting problems during the training process, we introduce regularization techniques into the network, thereby enhancing the robustness of the model. To further explore the interpretability of the model, we conduct a detailed analysis of the feature learning process of the network through visualization experiments, especially the visualization of feature distribution through UMAP, which clearly demonstrates the significant advantages of the algorithm in feature discrimination ability and verifies the effectiveness and adaptability of RMETNet in subject feature extraction.

[0215] These results indicate that RMETNet not only demonstrates excellent performance in the motor imagery classification task, but also provides new ideas and methods for cross-subject EEG signal classification.

[0216] In summary:

[0217] The classification model proposed by the present invention is based on TSLANet and Riemannian geometric features, and is an end-to-end neural network model called RMETNet (Riemannian MMD-enhanced EEG TSLANet). By introducing TSLANet, the decoding of complex temporal patterns of EEG signals is achieved. Subsequently, a time-frequency convolutional module is constructed to learn the time-frequency features of the signals. At the same time, Riemannian geometric features are fused to learn the distribution between subjects, obtaining more discriminative distribution features. The features are sent to the classifier to obtain the classification results. Meanwhile, by introducing a domain adaptation method based on the maximum mean discrepancy (MMD) loss, we align the feature distributions of the source domain and the target domain during the training process to reduce the influence of individual differences between subjects on the model, thereby verifying the effectiveness of the model in cross-subject EEG signal classification. The results show that the cross-subject experiment of this model achieved an average accuracy of 71.39% on the four-class BCI Competition IV 2a dataset, and the single-subject experiment on this dataset reached 80.71%. The cross-subject experiment on the two-class BCI Competition 2b dataset reached an average accuracy of 80.93%, and the single-subject experiment result reached 86.76%. Our results were the best in the comparison experiments with other classification algorithms. We also verified the feasibility of the proposed model through ablation experiments and discriminative feature visualization means. The experimental results show that the proposed method can effectively decode the feature information of EEG, reduce the distribution difference of cross-subject data features, and can be well applied to MI-EEG decoding research.

[0218] For those skilled in the art, it is obvious that the present invention is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or basic characteristics of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-limiting. The scope of the present invention is defined by the appended claims rather than the above description. Therefore, all changes falling within the meaning and scope of the equivalent elements of the claims are intended to be embraced within the present invention. Any reference signs in the claims should not be construed as limiting the claims involved.

Claims

1. A method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features, characterized by: At least the following steps are included: S1: Build a TSLANet module, which is used to effectively extract complex time features in the time-frequency space. The TSLANet module includes an adaptive spectrum module and an interactive convolution block, namely an ASB module and an ICB module. The ASB module and the ICB module form a single layer that can be extended to multiple layers. S2: Building a time-frequency convolution module, which is used to enable the model to have the ability to learn local time features in subsequent model building; S3: Building a multi-scale Riemann convolution module, which is used to process the Riemannian geometry features of the EEG signal, by calculating the covariance matrix of the source domain and the target domain data, mapping them to the Riemannian geometry space, and obtaining the feature representation after Riemannian geometry alignment; S4: Build a classifier module, use two fully connected layers as the classifier model, output an M-dimensional vector after the Softmax function, and use cross entropy as the loss function: Where M represents the number of EEG categories; y and are the true value and the predicted label respectively; N b represents the number of trials in the batch; S5: Based on the TSLANet module, the time-frequency convolution module, the multi-scale Riemann convolution module and the classifier module, a RMETNet model for realizing EEG signal classification is built. The RMETNet model is an end-to-end network module; S6: Train the RMETNet model, use the unsupervised distribution alignment method, and use the trained RMETNet model to classify motor imagery EEG signals across subjects.

2. The method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features according to claim 1, characterized in that: The ASB module uses Fourier transform to convert EEG time series data into frequency domain data, applies adaptive threshold to attenuate high-frequency noise and highlight relevant spectral features in EEG signals, and reconstructs time domain features using inverse fast Fourier transform after processing, thereby reducing noise and enhancing time series representation; The ICB module is a lightweight convolutional block that uses different kernel sizes to interactively refine features, thereby improving the adaptability to dynamic changes in time series and dynamically unifying the local and global time-varying feature extraction of time series analysis.

3. The method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features according to claim 1, characterized in that: The TSLANet module is composed of stacked TSLANets; Input multi-channel EEG signals are embedded in the layer to obtain deep features Where T is the time step; d model is the feature dimension; is the initial representation of the signal input to the TSLANet module; For the Lth layer TSLANet, the input is Then extract the deep temporal changes of the EEG signal: Specifically, TSLANet includes the following processes: First, the spectral features of each time series channel are obtained through fast Fourier transform, thereby generating a comprehensive frequency domain representation F, that is, the frequency domain data of the original EEG, for the one-dimensional time series features of the input The frequency domain representation is obtained by performing a fast Fourier transform along the spatial dimension, calculated as follows: Wherein, FTT[] represents the calculation of one-dimensional fast Fourier transform; L` represents the length of the EEG signal frequency domain sequence; Indicates that the frequency domain feature is in the form of a complex number; C represents the number of channels of the signal; Each channel of the signal is transformed independently to produce a comprehensive frequency domain representation F, which encompasses the spectral characteristics of the original EEG on all channels; Secondly, after obtaining the high-frequency noise, the high-frequency component usually indicates the fluctuation of the signal that deviates from the underlying trend. Here, the adaptive local filter is introduced, which can dynamically adjust the filtering level and remove the high-frequency noise component according to the data characteristics. It plays an important role in processing non-stationary EEG signals, because the non-stationary characteristics make its spectrum change rapidly over time, and the filter will adaptively set the frequency threshold; The signal power spectrum of F is calculated, which helps to identify the main frequency components in the EEG signal. The power spectrum density is calculated as the square of the frequency component amplitude: P = |F| 2 , provides a measure of the strength of different frequencies in the EEG signal sequence; The key to achieving effective EEG noise reduction is to adaptively filter out high-frequency components from the power spectrum P, which is achieved through a trainable threshold θ, which is adjusted according to the spectral characteristics of the data. The threshold θ is set as a learnable parameter so that it can be optimized by back propagation during the training process. Specifically, Represents the loss function The gradient of θ enables θ to identify basic signal frequencies and noise. The adaptive threshold formula is expressed as follows: F filtered =F⊙(P>θ) Among them, ⊙ represents element-by-element multiplication; (P>θ) is a binary mask, and all frequencies above the threshold θ are retained. This step ensures that high-frequency noise is effectively removed while retaining key information. By adaptively selecting the threshold, the adaptive spectrum module can customize the filtering process according to each EEG signal data, improving the effectiveness of the model in processing EEG data; After adaptively filtering the high-frequency data noise, two sets of learnable filters are adopted. A global filter is learned from the original EEG frequency domain data F, and a local filter is learned from the adaptively filtered EEG frequency domain data F. filtered Learning in Let W G and W L They are represented as learnable global filters and local filters respectively, and the formulas are as follows: F G =W G ⊙F F L =W L ⊙F filtered , Then the filtered features are integrated to capture all the features, i.e., F intergrated =F G +F L ; After obtaining all the features, the EEG frequency domain features are converted back to the time domain, and the inverse fast Fourier transform is applied to obtain the time domain signal X 1D `, the calculation formula is as follows: Inverse fast Fourier transform ensures that the structure of the enhanced EEG feature domain is consistent with the original input EEG signal data; After enhancing the feature representation through adaptive frequency and module, an interactive convolution module is used to capture local features and longer-distance dependencies, specifically: In the first convolutional layer, we try to use a small convolution kernel size to capture fine-grained local features in the EEG data, and use a larger convolution kernel in the second layer to identify broader and longer-range dependencies. Given the output X of the fast Fourier inverse transform 1D `, and use it as the input of the interactive convolution module. The process is as follows: A1=φ(Conv1(X 1D `))⊙Conv2(X 1D `) A2=φ(Conv2(X 1D `))⊙Conv1(X 1D `) Among them, Conv1(*) and Conv2(*) are two one-dimensional convolution operations; φ is the GELU activation function; and then pass through the final convolution layer Conv3(*): <h2 style=";text-align:left;direction:ltr">X<h2 style=";text-align:left;direction:ltr"> 1D <h2 style=";text-align:left;direction:ltr"> =Conv3(A1+A2) Among them, the output X 1D It represents the enhanced EEG signal features prepared for the last layer of the TSLANet module, and then the features of the TSLANet module are input into the following time-frequency convolution module.

4. The method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features according to claim 3, characterized in that: The time-frequency convolution module uses two convolutional layers in series and adopts a two-dimensional convolution kernel (1, K e ) to capture time information, and use express; A two-dimensional convolution kernel (C`, 1) is used to capture spatial information. Indicates that the size of C' is related to the channel of the EEG signal. The spatial filter is learned through spatial convolution, which is related to the EEG channel. Such a structure can effectively extract the local information of the EEG signal in the time dimension and the global features in the spatial dimension. Let X 1D The 1D feature samples output by the TSLANet module are expanded into 2D samples using the following formula: X 2D =ExpendDim(X 1D ,1) Among them, ExpendDim(*,1) means to expand the dimension in the second dimension, that is, from become This operation enables the features to match the two-dimensional convolution kernel; in order to maintain the non-stationary characteristics of the signal, the square activation function is used for processing; in order to reduce noise and the number of model parameters, the pooling layer is used to downsample the feature map; in order to return the average value of the potential features to zero, improve training stability, and speed up the training process, batch normalization is performed after the activation function; Use X 2D represents the output of the convolution operation, Where b is the number of samples, f is the number of channels of the output feature map of the convolution module, c is the number of channels, and s is the number of sample features, which are defined as follows: in, and Represents the size of the two convolution kernels; X 2D is the input feature; Conv2D() indicates a two-dimensional convolution operation using two convolution kernels; Square() indicates a square activation function; AvgPool() indicates an average pooling operation; In order to ensure the stability of feature distribution and avoid the problem of internal covariate shift in the model, the logarithmic activation function and batch normalization operation are added after the square activation function, which are defined as follows: X 2D =BatchNorm(Log(X′ 2D )) Among them, X′ 2D It is the output feature after three convolution and average pooling operations, Log() represents the logarithmic activation function, and BatchNorm() represents the batch normalization operation.

5. The method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features according to claim 4, characterized in that: The original EEG signal input to the multi-scale Riemann convolution module is First, the covariance matrix is ​​calculated as follows: Among them, Cov represents the covariance matrix between channels, that is, the covariance matrix after the source domain signal data and the target domain signal data are calculated. Then, the Riemannian geometric distance between the two matrices is calculated using the Riemannian geometric distance, which is defined as follows: Among them, log(Cov1) represents the matrix logarithm of the covariance matrix. Through eigenvalue calculation, all covariance matrices are mapped to a reference point in Riemann geometry, and the geometric mean is calculated as Each covariance matrix Cov i Pan to reference point The calculation formula is as follows: A specific structure is designed for the aligned covariance matrix to extract multi-scale features and improve the robustness of data points. Three convolutional layers with different kernel sizes are used to extract features at three levels: local range, medium scale, and global distribution model, and then feature fusion is performed. Finally, flatten it into a low-dimensional feature space to generate a compact feature representation: X rout =Linear(Flatten(X cat )) Then, the features output by the multi-scale Riemann module are fused with the time-frequency convolution module, and the model is sent to the classifier module.

6. The method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features according to claim 5, characterized in that: The input of the RMETNet model is the preprocessed motor imagery EEG signal data, and the RMETNet model includes two branches; The first branch is input by the original EEG data, and the TSLANet module is used to capture the temporal feature pattern of the EEG signal. Then, the time series features with higher-level representation are output through the time convolution and depth convolution in the time-frequency convolution module. The second branch takes the Riemann features calculated from the source domain and the target domain as input. The second branch extracts comprehensive Riemannian geometric features through a multi-scale Riemannian convolution module as different cross-subject distribution features; Finally, the features of the two branches are combined and sent to the fully connected layer with SoftMax classifier, i.e., the classifier module, to obtain the probability result of motor imagery EEG signal classification.

7. The method for classifying EEG signals of cross-subject motion phenomena based on TSLANet and Riemannian geometry features according to claim 1, characterized in that: The unsupervised distribution alignment method uses MMD loss to calculate the feature distribution difference as domain alignment loss, quantifies the difference between the source domain and the target domain, can reduce the distribution difference between the source domain and the target domain, align their global distribution, and realize the cross-domain features by sharing network parameters. The process is defined as follows: in represents the classification loss; λ represents the pair, which is calculated from the source domain label; Represents the MMD loss, which minimizes the distribution difference between the feature classifications of the source domain and the target domain. The calculation formula is as follows: where F s ,F t Represents the source domain data X s and target domain data X t Input the feature representation obtained by the network.

Images