Electroencephalogram signal classification method and system based on mixed manifold attention network

Abstract

The application relates to the technical field of electroencephalogram signal classification, and discloses an electroencephalogram signal classification method and system based on a mixed manifold attention network, which comprises the following steps: constructing a mixed manifold attention network model, wherein the model comprises a feature extraction module, a manifold modeling module and a mixed manifold modeling module; the manifold modeling module divides the space-time semantic features of an EEG signal in two dimensions of time and channel to obtain space-time and space-time data, and models the space-time and space-time data to SPD manifolds and Grassmann manifolds; the mixed manifold modeling module comprises an SPD manifold self-attention module, a Grassmann manifold self-attention module and a mixed Riemann similarity measurement module, which are respectively used for extracting SPD manifold features based on LCM, extracting Grassmann manifold features and integrating geodesic distances on different manifolds through weighted summation; and the model is used for classifying EEG signals. The application can mine the geometric correlation of signals, improve the robustness, calculation efficiency and accuracy of the model.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) signal classification technology, and in particular to an EEG signal classification method and system based on a hybrid manifold attention network. Background Technology

[0002] With the development and advancement of science and technology, brain-computer interfaces (BCIs), as an advanced human-computer interaction technology, have become an effective tool for non-invasive monitoring and regulation of brain function due to their ability to directly connect to external devices through brain electrical activity. They are widely used in numerous fields such as medicine, rehabilitation, and intelligent control. Electroencephalography (EEG) signals, with their non-invasive nature, have been widely applied in BCI systems, especially in steady-state visual evoked potential paradigms and error-correlated negative wave paradigms, where they can accurately extract and decode brain responses and feedback. However, the non-linear and constantly changing structure of EEG, coupled with its sensitivity to external disturbances, makes EEG data highly complex. Extracting key information from complex EEG data has become a challenging problem. Furthermore, achieving robust representation and real-time decoding of EEG data while maintaining classification accuracy is also a technical challenge.

[0003] Traditional EEG signal decoding and analysis typically rely on feature engineering and shallow learning models, such as traditional methods based on time-domain or frequency-domain features, which manually extract EEG statistical features for classification and recognition. These methods achieved good progress in early brain-computer interface applications, but their limitations have become increasingly apparent as the complexity of EEG data continues to increase. Furthermore, traditional methods cannot capture effective statistical features from complex EEG activity signals, especially when multiple EEG rhythms are intertwined; the nonlinearity and dynamics of the signal make feature extraction even more difficult. Simultaneously, these methods are highly sensitive to noise and external interference, leading to unstable classification performance. Therefore, with the increasing demands for robustness and real-time performance in EEG signal decoding, traditional methods still face significant challenges in terms of efficiency and accuracy when processing large-scale and high-dimensional EEG data. Specifically, the shortcomings of existing technologies in processing EEG signals include:

[0004] (1) Ineffective representation of the non-Euclidean structure of EEG data. Most existing methods rely on Euclidean neural networks, such as Convolutional Neural Networks (CNNs) and Transformers, for non-linear feature learning, neglecting the inherent low-dimensional submanifold structure of sequence data like EEG. In recent years, Riemannian geometry-based data representation methods have proven widely effective in modeling the non-Euclidean structure of data in tasks such as image processing and computer vision. However, there is currently no effective approach to encoding and learning EEG data using Riemannian geometry theory.

[0005] (2) It cannot effectively encode the spatiotemporal features of EEG data. The complex spatiotemporal variation features contained in EEG data are one of the key factors enabling brain-computer interface applications to achieve performance breakthroughs. Compared with EEG classification methods in Euclidean space, although non-Euclidean classification methods represented by Riemannian manifolds can improve accuracy to a certain extent, this method is an implicit, coarse-grained EEG data modeling mechanism that cannot explicitly and finely mine and encode the geometric correlation of EEG signals in the spatiotemporal dimension.

[0006] (3) The existing manifold-based classification methods cannot effectively improve classification performance. Most Riemannian manifold-based EEG classification algorithms are typically built on a single symmetric positive definite (SPD) manifold, failing to fully explore and utilize the statistical complementarity between heterogeneous manifolds. Furthermore, the matrix eigenvalue functions involved in Riemannian geometric operators and the inability of graphics processing units (GPUs) to accelerate them also affect the inference speed of SPD algorithms. Therefore, existing technologies cannot effectively integrate the advantages of different matrix manifolds, and cannot improve the efficiency and robustness of EEG data decoding. Summary of the Invention

[0007] Therefore, the technical problem to be solved by the present invention is to overcome the shortcomings of the prior art and provide a method and system for classifying EEG signals based on a hybrid manifold attention network, which can explicitly and finely mine and encode the geometric correlation of EEG signals in the spatiotemporal dimension, effectively maintain the Riemannian geometric structure, and improve the robustness, computational efficiency and accuracy of the model.

[0008] To address the aforementioned technical problems, this invention provides a method for classifying EEG signals based on a hybrid manifold attention network, comprising the following steps: acquiring EEG signals and dividing them into training and testing sets; constructing a hybrid manifold attention network model, wherein the hybrid manifold attention network model includes a feature extraction module, a manifold modeling module, and a hybrid manifold modeling module; the feature extraction module extracts the spatiotemporal semantic features of the EEG signals; the manifold modeling module divides the spatiotemporal semantic features of the EEG signals in both time and channel dimensions to obtain spatiotemporal token data and spatiotemporal token data respectively; and then... The data is modeled simultaneously onto the SPD manifold and the Grassmann manifold. The hybrid manifold modeling module includes an SPD manifold self-attention module, a Grassmann manifold self-attention module, and a hybrid Riemann similarity metric module. The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, the Grassmann manifold self-attention module extracts Grassmann manifold features, and the hybrid Riemann similarity metric module integrates geodesic distances on different manifolds through weighted summation. The hybrid manifold attention network model is trained using the training set, and the EEG signal classification results are obtained by inputting the test set into the trained hybrid manifold attention network model.

[0009] Furthermore, the process of dividing the spatiotemporal semantic features of the EEG signal into spatiotemporal token data and spatiotemporal token data in both time and channel dimensions is as follows:

[0010] The spatiotemporal semantic features of the EEG signal are denoted as feature matrix E. i E i The size is c×l, for the characteristic matrix E i Fine-grained segmentation is performed along both the channel and time dimensions:

[0011] In the spatial dimension, E i The data is divided into m continuous and non-overlapping segments, each segment having a size of h×l, where h = c / m; the resulting space-time token data is used... express, This represents the j-th empty time token data, where j = 1, ..., m;

[0012] In the time dimension, E i Divide the data into n consecutive and non-overlapping segments, each segment having a size of c × d, where d = l / n; This indicates the obtained spatiotemporal token data. This represents the k-th spatiotemporal token data, where k = 1, ..., n.

[0013] Furthermore, the step of simultaneously modeling spatiotemporal token data and spacetime token data onto the SPD manifold and Grassman manifold specifically involves:

[0014] The SPD manifold representation corresponding to the spatiotemporal token data and the spatiotemporal token data is as follows:

[0015] For the j-th space-time token data, the formula for calculating the covariance is:

[0016]

[0017] in, express The covariance characterization, express The eigenvector of the t-th column in the vector, t = 1, ..., l. express The column mean;

[0018] For the k-th spatiotemporal token data, the formula for calculating the covariance is:

[0019]

[0020] in, express The covariance characterization, express The eigenvectors in the r-th row of the vector, where r = 1, ..., c. express The row mean;

[0021] Use regularization techniques to ensure and All are SPD matrices:

[0022]

[0023] Where I represents the identity matrix, Trace(·) refers to the trace of the matrix, and λ is a perturbation constant;

[0024] The Grassman manifold representations corresponding to spatiotemporal token data and spacetime token data are obtained by using the singular value decomposition of the matrix, specifically:

[0025] The j-th spacetime token data and k spatiotemporal token data The corresponding subspace is represented as:

[0026]

[0027] in, They represent The corresponding subspace representation, They represent The eigenvector matrix corresponding to the first q largest eigenvalues, and This is the corresponding eigenvalue matrix, where T represents the transpose;

[0028] At this moment, the original feature matrix E i The corresponding hybrid manifold representations are expressed in the following set language as follows:

[0029]

[0030] in, and They refer to E respectively i The corresponding SPD manifold representation and Grassmann manifold representation, due to and and All have the same size, and for ease of subsequent description, the following simplified data representation form is adopted: Where s = m + n.

[0031] Furthermore, the SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, specifically as follows:

[0032] The query Q, key K, and value V on the SPD manifold are generated using a bilinear mapping, as shown below:

[0033]

[0034] Among them, Q r K r V r Let Q, K, and V represent the feature matrices corresponding to the r-th query Q, key K, and value V on the SPD manifold, respectively. (d t ≤d) are three projection matrices that satisfy column orthogonality;

[0035] Q is obtained using Cholesky decomposition. r K r V r The corresponding query in the Choreskiy manifold key Sum Recorded as The calculation formula is as follows:

[0036]

[0037] in, These are the three lower triangular matrices obtained;

[0038] Using LCM calculation and The geodesic distance between them is calculated using the following formula:

[0039]

[0040] in, for and Geodesic distances between them, r = 1, ..., s, j = 1, ..., s, Reference A strictly lower triangular matrix. Indicates by The diagonal matrix formed by the main diagonal; Log represents the logarithm of the matrix.

[0041] calculate The attention weight coefficient is calculated using the following formula:

[0042]

[0043] in, for Attention weight coefficients; This represents the attention weight matrix output by the self-attention module of the SPD manifold;

[0044] The weighted Frescher mean is used to aggregate features in a manifold, and the calculation formula is as follows:

[0045]

[0046] in, For feature aggregation in a manifold, Representing the Choreskiy manifold, Referring to the Riemannian metric on the Choreskiy manifold, This represents the mean of the manifold that needs to be optimized. The closed-form solution is shown below:

[0047]

[0048] For the solution The inverse operation of the Cholliski decomposition is used to map it back to the SPD manifold, and the calculation formula is as follows:

[0049]

[0050] Among them, C′ r for The representation after mapping back to the SPD manifold;

[0051] At this point, one attention calculation process on the SPD manifold has been completed, and the resulting new set of SPD data points is represented as follows: That is, the SPD manifold characteristics.

[0052] Furthermore, the Grassmann manifold self-attention module extracts Grassmann manifold features, specifically as follows:

[0053] for The query Q, key K, and value V on the Grassman manifold are calculated using the following formula:

[0054]

[0055] in, Let Q, K, and V represent the feature matrices corresponding to the r-th query Q, key K, and value V on the Grassman manifold, respectively. (d t ≤d) are three projection matrices that satisfy column orthogonality;

[0056] right Perform the QR decomposition as shown below:

[0057]

[0058] in, It is an orthogonal matrix. It is an upper triangular matrix; use the following formula to... The transformation into an eigenma matrix with column orthogonality is shown below:

[0059]

[0060] in, for The transformed feature matrix with column orthogonality;

[0061] Use and Calculation Obtained in the same way and

[0062] Calculation using OGIRM and The geodesic distance between them is calculated using the following formula:

[0063]

[0064] in, for and Geodesic distances between them, r = 1,…,s, j = 1,…,s; ∑ a Let be the a-th main diagonal element in ∑, where a = 1, ..., q; thinSVD represents the thin singular value decomposition of a matrix, i.e. θ a Let Θ be the a-th main diagonal element, and Θ = cos -1 ∑;

[0065] Will The formula for converting to attention weight coefficients is shown below:

[0066]

[0067] Where, η rj for Attention weight coefficients, using This represents the attention weight matrix output by the self-attention module of the Grassman manifold;

[0068] Using the orthogonal group invariant Riemannian metric OGIRM from the perspective of ONB geometry, and its associated exponential and logarithmic mappings, we calculate the intrinsic mean on the Grassmann manifold.

[0069] At this point, one attention calculation process on the Grassmann manifold has been completed, and the resulting new set of subspace data points is represented as follows: That is, the SPD Grassmann manifold feature.

[0070] Furthermore, the intrinsic mean on the Grassmann manifold is calculated using the orthogonal group-invariant Riemannian metric OGIRM from the ONB geometric perspective, along with its associated exponential and logarithmic mappings. The calculation process is as follows:

[0071] Choose an orthogonal basis matrix from the orthogonal complement space of all-one vectors. use Each input Project to 1 ⊥ , represented as:

[0072] For E j Perform the thin singular value decomposition operation:

[0073]

[0074] Wherein, thinSVD represents the thin singular value decomposition of the matrix. Logarithmic mapping and weighted average will be used to... Perform iterative calculations to obtain the results.

[0075] Will Projecting back into the original space, the result is denoted as Y′. r ,Y′ r The calculation method is as follows:

[0076]

[0077] Furthermore, the method of logarithmic mapping and weighted averaging will... Perform iterative calculations, specifically:

[0078] Will As the initial value for the mean, repeat the following steps:

[0079] Let the current iteration be the t-th iteration. Using the logarithmic mapping GLog, we map the data to the tangent space of the result calculated in the previous iteration, denoted as π. j , π j Specifically:

[0080]

[0081] in, This is the result obtained from the (t-1)th iteration. ρ=tan -1 θ,

[0082] Calculate the arithmetic weighted average of the samples in the tangent space, denoted as τ. The method for calculating τ is as follows:

[0083]

[0084] The obtained τ is transformed back to the Grassmann manifold using the exponential mapping GExp:

[0085]

[0086] Where, μθδ T =thin SVD(τ);

[0087] Repeat the above iterative calculation process until the iteration stopping condition is met, and then stop the iteration calculation. As

[0088] Furthermore, the hybrid Riemann similarity metric module integrates geodesic distances on different manifolds through weighted summation, and the calculation formula is as follows:

[0089]

[0090] in The values ​​represent the attention weight coefficients output by the hybrid Riemann similarity metric module, r = 1, ..., s, j = 1, ..., s. ξ1 and ξ2 are equilibrium parameters;

[0091] make Implement dynamic feature interaction based on attention matrix.

[0092] Furthermore, the hybrid manifold attention network model also includes a signal classification module, specifically:

[0093] An embedded mapping layer, denoted as SMEem, is constructed at the output of the SPD manifold self-attention module, and an embedded mapping layer, denoted as GMEem, is constructed at the output of the Grassmann manifold self-attention module.

[0094] Using the logarithmic mapping function induced by LCM, the SMEem layer is calculated as follows:

[0095]

[0096] in, Denotes a symmetric matrix located in Euclidean space;

[0097] The calculation formula for the GMEem layer is as follows:

[0098]

[0099] in, This indicates the truncated identity matrix. Represents a column-full-rank matrix in Euclidean space;

[0100] Construct a hybrid feature fusion layer, and output a Euclidean feature vector, denoted as ψ, through feature concatenation. r , ψ r The calculation formula is:

[0101]

[0102] in, Indicates taking out The lower triangular portion is then stretched into a column vector. Indicates will To stretch the vector into column vectors, `Concat(,)` refers to concatenating features along the column direction.

[0103] The FC layer and Softmax function are embedded at the end of the hybrid feature fusion layer to achieve EEG signal classification.

[0104] This invention also provides an EEG signal classification system based on a hybrid manifold attention network, comprising:

[0105] The data acquisition module is used to acquire EEG signals and divide them into training and test sets.

[0106] A hybrid manifold attention network model construction module is used to construct a hybrid manifold attention network model. The hybrid manifold attention network model includes a feature extraction module, a manifold modeling module, and a hybrid manifold modeling module. The feature extraction module extracts the spatiotemporal semantic features of the EEG signal. The manifold modeling module divides the spatiotemporal semantic features of the EEG signal in both time and channel dimensions to obtain spatiotemporal token data and space-time token data, and simultaneously models the spatiotemporal token data and space-time token data onto the SPD manifold and the Grassmann manifold.

[0107] The hybrid manifold modeling module includes an SPD manifold self-attention module, a Grassman manifold self-attention module, and a hybrid Riemann similarity measurement module. The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, the Grassman manifold self-attention module extracts Grassman manifold features, and the hybrid Riemann similarity measurement module integrates geodesic distances on different manifolds through weighted summation.

[0108] The training module is used to train the hybrid manifold attention network model using the training set;

[0109] The classification module is used to input the test set into the trained hybrid manifold attention network model to obtain the classification results of the EEG signals.

[0110] Compared with the prior art, the above-described technical solution of the present invention has the following advantages:

[0111] This invention extracts complementary Riemannian geometric representations through a manifold modeling module, which can more effectively preserve the Riemannian geometric structure of the input data. The SPD manifold self-attention module is built based on the log-Cholsky metric, which improves the robustness and computational efficiency of the model. By combining the feature extraction module, manifold modeling module, and hybrid manifold modeling module, the geometric correlation of EEG signals in the spatiotemporal dimensions is explicitly and finely mined and encoded. The hybrid Riemann similarity metric module realizes the accurate determination of geometric similarity between manifold data, thereby improving the model's learning ability of EEG signals and increasing accuracy. Attached Figure Description

[0112] To make the content of this invention easier to understand, the invention will be further described in detail below with reference to specific embodiments and accompanying drawings, wherein:

[0113] Figure 1 This is a flowchart of a method in a preferred embodiment of the present invention.

[0114] Figure 2 This is an overall architecture diagram of the hybrid manifold attention network model constructed in a preferred embodiment of the present invention.

[0115] Figure 3 This is a schematic diagram of the hybrid manifold modeling process in a preferred embodiment of the present invention.

[0116] Figure 4 This diagram illustrates the specific implementation process of the hybrid manifold attention network model constructed in a preferred embodiment of the present invention. Detailed Implementation

[0117] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0118] Reference Figure 1 , Figure 2 As shown, this invention discloses a method for classifying electroencephalogram (EEG) signals based on a hybrid manifold attention network, comprising the following steps:

[0119] S1: Acquire EEG signals and divide them into training and test sets.

[0120] S2: Construct a Hybrid Manifold Attention Network (HMANet) model, which includes a Feature Extraction Module (FEM) based on Euclidean convolution, a manifold modeling module based on subspace descriptors, a hybrid manifold modeling module, and an EEG signal classification module based on feature fusion (FFCM). The Feature Extraction Module extracts the spatiotemporal semantic features of the EEG signal. The Manifold Modeling Module divides the spatiotemporal semantic features of the EEG signal in both time and channel dimensions to obtain a set of spatiotemporal token data and a set of spacetime token data. The spatiotemporal token data and the spacetime token data are simultaneously modeled onto the SPD manifold and the Grassmann manifold to achieve complementary Riemannian geometric representations.

[0121] S2-1: The Feature Extraction Module (FEM) contains two convolutional blocks, Conv_block1 and Conv_block2. The convolutional operations in the first Conv_block are used to extract spatial features from the multi-channel EEG signal, while the convolutional operations in the second Conv_block are used to extract spatiotemporal features. Each convolutional layer is followed by an ELU activation function to enhance the model's nonlinear representation capability.

[0122] Let X be an input EEG signal sequence. i (Assuming its initial size is e×d, and the i-th input EEG signal sequence), it is mapped to a spatiotemporal semantic space, and the resulting new data is represented by a feature matrix E of size c×l. i This process can be represented as: E i =φFEM (X i ).

[0123] S2-2: In order to effectively mine and utilize the rich spatiotemporal information contained in EEG signals, the spatiotemporal semantic features of EEG signals are denoted as feature matrix E. i E i The size is c×l, for the characteristic matrix E i Fine-grained segmentation is performed along both the channel and time dimensions:

[0124] In the spatial dimension, E i The data is divided into m continuous and non-overlapping segments, each segment having a size of h×l, where h = c / m; at this point, the resulting set of space-time token data is used... express, This represents the j-th empty time token data, where j = 1, ..., m;

[0125] In the time dimension, E i Divide the data into n consecutive and non-overlapping segments, each segment having a size of c × d, where d = l / n; at this point, use... This represents the obtained set of spatiotemporal token data. Let h represent the k-th spatiotemporal token data, where k = 1, ..., n. To simplify subsequent self-attention calculations, h and d are set to the same value in this invention.

[0126] S2-3: The process of simultaneously modeling spatiotemporal token data and spatiotemporal token data onto the SPD manifold and Grassman manifold is as follows:

[0127] S2-3-1: The SPD manifold representation corresponding to the spatiotemporal Token data and the spatiotemporal Token data is as follows:

[0128] For the j-th spacetime token data The formula for calculating covariance is:

[0129]

[0130] in, express The covariance characterization, express The eigenvector of the t-th column in the vector, t = 1, ..., l. express The column mean; The calculation formula is:

[0131]

[0132] Similarly, for the k-th spatiotemporal token data The formula for calculating covariance is:

[0133]

[0134] in, express The covariance characterization, express The eigenvectors in the r-th row of the vector, where r = 1, ..., c. express The row mean; The calculation formula is:

[0135]

[0136] Since the fundamental elements in an SPD manifold are SPD matrices, the resulting and It is not necessarily positive definite. Therefore, using a common regularization technique as shown below can ensure that... and All are SPD matrices:

[0137]

[0138] Here, I represents the identity matrix, Trace(·) refers to the trace of the matrix, and λ is a perturbation constant; its value is generally selected from the set {1e-3, 1e-4, 1e-5}. If λ is taken as a large value, it will introduce a large perturbation into the eigenspace of the input matrix; if the value of λ is set to a small value, the input matrix will be close to ill-conditioned (positive semi-definite). Both of these situations will affect the stability of the classification system.

[0139] S2-3-2: Based on the above formulas (1) and (3), the Grassmann manifold representations corresponding to the spatiotemporal token data and the spatiotemporal token data can be obtained by using the singular value decomposition (SVD) of the matrix, specifically:

[0140] The j-th spacetime token data and k spatiotemporal token data The corresponding subspace is represented as:

[0141]

[0142] in, They represent The corresponding subspace representation, They represent The eigenvector matrix corresponding to the first q largest eigenvalues, and This is the corresponding eigenvalue matrix, where T denotes the transpose; since the fundamental elements in a Grassmann manifold are equivalence classes (in which an h×q column orthogonal matrix is ​​equal to another column orthogonal matrix of the same size multiplied by a q×q orthogonal matrix), and The column vectors then form an orthonormal basis for each other in their respective q-dimensional subspaces.

[0143] S2-3-3: At this moment, the original feature matrix E i The corresponding hybrid manifold representations are expressed in the following set language as follows:

[0144]

[0145] in, and They refer to E respectively i The corresponding SPD manifold representation and Grassmann manifold representation, due to and and All have the same size (d = h). For ease of subsequent description, the following simplified data representation is adopted: Where s = m + n.

[0146] Existing Riemannian deep learning-based EEG signal classification algorithms fail to fully exploit the statistical correlations of EEG signal sequences across the spatiotemporal dimensions, relying solely on a single Riemannian manifold category (SPD manifold or Grassmann manifold) and neglecting the complementary nature of heterogeneous manifolds in Riemannian geometry. The hybrid manifold modeling module simultaneously models spatiotemporal token data onto both SPD and Grassmann manifolds, as follows: Figure 3 As shown, the hybrid manifold modeling in this invention not only achieves spatiotemporal-space-time joint mining of EEG signal data, but also realizes statistical complementary representation of the intrinsic submanifold structure of the data. This explicit, fine-grained spatiotemporal statistical representation mechanism provides more diverse input information for the subsequent attention calculation process, enhancing the model's discriminative learning ability.

[0147] S2-4: To reduce computation time, this invention designs a novel SPD manifold self-attention module (LCMANet) using Riemannian geometric operators such as exponential mapping, logarithmic mapping, and weighted Fréchet mean associated with LCM. Since the geometric operators under LCM all have simple mathematical expressions and closed-form solutions (not involving any form of matrix eigenvalue operation), LCMANet exhibits significantly higher computational efficiency. Furthermore, compared to the quadratic dependency of LEM on the SPD matrix, LCM is approximately linear (the quadratic dependency only applies to the main diagonal elements of the SPD matrix), resulting in a more stable numerical computation process for LCMANet.

[0148] The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, specifically:

[0149] S2-4-1: Due to Since the data is represented on the SPD manifold, rather than as a data vector in Euclidean space, traditional linear mapping mechanisms are not applicable. Therefore, a bilinear mapping (defined as the Sbm layer in LCMANet for ease of description) is first used to generate the query Q, key K, and value V on the SPD manifold, as shown below:

[0150]

[0151] Among them, Q r K r V r Let Q, K, and V represent the feature matrices corresponding to the r-th query Q, key K, and value V on the SPD manifold, respectively. (d t ≤d) are three projection matrices that satisfy column orthogonality (located on a Stiefel manifold); the resulting characteristic matrix They still possess positive definiteness, meaning they remain elements on the SPD manifold.

[0152] S2-4-2: Next, Q is obtained using the Cholesky decomposition (defined as the Cholesky layer in LCMANet for ease of description). r K r V r The corresponding query in the Choreskiy manifold key Sum Recorded as The calculation formula is as follows:

[0153]

[0154] in, These are the three lower triangular matrices obtained; since the input data for the Chol layer is the SPD matrix, therefore The main diagonal elements are all positive numbers, and the above decomposition is unique. Furthermore, the Choreskii manifold (the space spanned by lower triangular matrices with positive main diagonals) is essentially a Euclidean space. A submanifold. Therefore, designing attention networks in the Choleski manifold not only enables more efficient encoding and learning of the submanifold structure of EEG data, but also avoids destroying the Riemannian geometric properties of the original data, since the Choleski manifold and the SPD manifold are diffeomorphic.

[0155] S2-4-3: One of the key steps in building an attention computation framework is measurement. and The similarity between them. Unlike traditional calculations based on Euclidean vector inner products, the operation here deals with two structured matrices located in a Chollesky manifold. Therefore, this invention first designs a network layer with a logarithmic Chollesky metric (LCM layer) to utilize LCM for calculation. and The geodesic distance between them is calculated using the following formula:

[0156]

[0157] in, for and Geodesic distances between them, r = 1, ..., s, j = 1, ..., s, Reference A strictly lower triangular matrix (excluding the main diagonal portion). Indicates by The diagonal matrix formed by the main diagonal, where Log represents the logarithm of the matrix.

[0158] S2-4-4: However, the result calculated according to formula (11) The main reason why it cannot be directly used as attention weight is that the similarity between samples is inversely proportional to the distance between samples. Therefore, this invention further designs a network layer called the similarity determination layer (abbreviated as Sim layer) to calculate... The attention weight coefficient is calculated using the following formula:

[0159]

[0160] in, for The attention weight coefficients; it can be seen that the transformation function f sim Its function is reflected in two aspects: First, It is about The monotonically decreasing function f can effectively measure the actual similarity between samples; secondly, f sim Introducing the Softmax function to compress the attention matrix in the row direction (using...) The range of values ​​of (represented by) is such that each row of its elements satisfies convex constraints, which is beneficial for system parameter optimization. In this case, using... This represents the attention weight matrix output by the self-attention module (Sim layer) of the SPD manifold.

[0161] S2-4-5: Another key step in constructing the attention computation framework is to obtain the optimized feature matrix using a weighted average (i.e., the feature aggregation process). Considering that the operation here involves the lower triangular matrix in the Choreski manifold rather than Euclidean vectors, this invention uses the weighted Fréchet Mean (wFM) to achieve feature aggregation in the manifold. The main reasons for choosing wFM are twofold: 1) wFM can effectively utilize and preserve the Riemannian geometry of the data; 2) Extensive research has shown that wFM has significant theoretical and practical advantages in manifold data analysis. For ease of explanation, this invention designs a network layer called wFM in LCMANet, and the calculation formula is as follows:

[0162]

[0163] in, For feature aggregation in a manifold, Representing the Choreskiy manifold, Referring to the Riemannian metric on the Choreskiy manifold, Let represent the mean of the manifold to be optimized. Since the present invention uses the log-Chollisky metric (LCM), the corresponding distance formula is shown in formula (11) above. Therefore, it can be directly derived from formula (13). The closed-form solution, The closed-form solution is shown below:

[0164]

[0165] As can be seen, the wFM based on LCM in this invention reduces the computational complexity to a minimum by restricting the logarithmic and exponential operations of the matrix to a diagonal SPD matrix. This significantly improves the system's computational efficiency.

[0166] S2-4-6: Finally, for the solved... This invention utilizes the inverse operation of the Choleski decomposition to map it back to the SPD manifold. For ease of explanation, the above operation is represented in LCMANet by a network layer named IChol, and its calculation formula is as follows:

[0167]

[0168] Among them, C′ r for The representation after mapping back to the SPD manifold.

[0169] At this point, one attention calculation process on the SPD manifold has been completed, and the resulting new set of SPD data points is represented as follows: This refers to the SPD manifold features. It can be seen that the Sbm layer, Chol layer, LCM layer, Sim layer, wFM layer, and IChol layer together constitute an attention computation module in LCMANet. Compared to Matt, LCMANet has advantages in two aspects: 1) LCMANet uses Cholesky decomposition and its inverse mapping to equivalently transfer the attention computation process on the SPD manifold to the Cholesky space (homeomorphic to the SPD manifold). Its Euclidean space embedding submanifold structure allows the proposed attention model to more effectively encode and learn the EEG data structure, improving the representation learning ability of the classification system; 2) Thanks to LCM, LCMANet's computation process does not involve complex matrix functions (such as SVD), requiring only one Cholesky decomposition and its inverse mapping. Compared to the Matt model, which requires at least 2s matrix SVDs, this invention significantly improves the computational efficiency of the EEG classification system.

[0170] S2-5: To effectively preserve the submanifold structure of the input data, this invention proposes a Grassman attention module (IRGANet) that enhances representation capabilities. It utilizes the Orthogonal Group Invariant Riemannian Metric (OGIRM) from the perspective of ONB, along with its associated exponential and logarithmic mappings, to give a mathematical definition of the intrinsic mean on the Grassman manifold. This more fully explores the Riemannian geometric structure of the data, thereby making the Riemannian mean obtained through iterative optimization more accurate and effective.

[0171] The Grassmann manifold self-attention module extracts Grassmann manifold features, specifically as follows:

[0172] S2-5-1: The existing Grassman manifold self-attention network GDLNet uses wFM based on projection metric (PM) to achieve feature aggregation on the manifold. However, the projection matrix to be solved... The constraint of idempotency (i.e., This constraint determines that it is a non-convex parameter optimization problem. Therefore, GDLNet ignores this constraint, allowing the parameters to be derived directly using a weighted average in Euclidean space. (Idempotency is not satisfied). Next, SVD decomposition is performed, and the eigenvector matrices corresponding to the top q largest eigenvalues ​​are extracted, thus deriving the Grassmann manifold mean, referred to as the extrinsic mean. It can be seen that the above approximate solution process cannot obtain the optimal mean point on the Grassmann manifold, thereby affecting the classification performance of the system. To alleviate this problem, this invention constructs a Grassmann manifold self-attention module that enhances representational capabilities. It defines the Riemann mean on the Grassmann manifold, referred to as the intrinsic mean, by utilizing the Orthogonal Group Invariant Riemannian Metric (OGIRM) from the ONB perspective and its associated exponential and logarithmic mappings. Compared to GDLNet, it can effectively encode and preserve the Riemannian geometry of the data manifold, improving the model's representational capabilities and thus making it suitable for more complex EEG signal classification tasks.

[0173] Specifically, for a set of sample points on a Grassmanifold output by a hybrid spatiotemporal modeling network This invention first generates a query Q, a key K, and a value V on a Grassman manifold. For ease of description, this invention defines a network layer in IRGANet called the Grassman projection mapping (abbreviated as Gpm layer). The specific calculation formula is as follows:

[0174]

[0175] in, Let Q, K, and V represent the feature matrices corresponding to the r-th query Q, key K, and value V on the Grassman manifold, respectively. (d t ≤d) are three projection matrices that satisfy column orthogonality (located on a Stiefel manifold); however, the resulting eigenmatrix They do not satisfy the column orthogonality, meaning they are not currently located in the Grassmann manifold.

[0176] S2-5-2: To address this, the present invention employs the following method: IRGANet includes a network layer called Orthogonal Regularization (abbreviated as Ore layer), which, through the QR decomposition of the matrix, "pulls" the aforementioned data back to the Grassmann manifold. For ease of description, it is referred to here as... Let's take an example to illustrate the calculation process of the Ore layer. First, for... Perform the QR decomposition as shown below:

[0177]

[0178] in, It is an orthogonal matrix. It is an upper triangular matrix. Since R r It is reversible, therefore the following formula can be used to... The transformation into an eigenma matrix with column orthogonality is shown below:

[0179]

[0180] in, for The transformed feature matrix with column orthogonality.

[0181] Use and Calculation Obtained in the same way and

[0182] S2-5-3: Subsequently, this invention first designed a network layer called the Orthogonal Group Invariant Riemann Metric (abbreviated as OgiRm layer), the purpose of which is to utilize OGIRM to compute... and The geodesic distance between them is calculated using the following formula:

[0183]

[0184] in, for and Geodesic distances between them, r = 1,…,s, j = 1,…,s; ∑ a Let be the a-th main diagonal element in ∑, where a = 1, ..., q; thinSVD represents the thin singular value decomposition of a matrix, i.e. θ a Let Θ be the a-th main diagonal element, and Θ = cos -1 Similarly, it can be seen that... It cannot be directly used as attention weights in IRGANet.

[0185] S2-5-4: To this end, similar to the previously proposed LCMANet, IRGANet further designed a network layer called the similarity determination layer (abbreviated as Gsim layer), the main purpose of which is to... The formula for converting this into an effective attention weight coefficient is shown below:

[0186]

[0187] Where, η rj for The attention weight coefficient, at this point, is used Let represent the attention weight matrix output by the self-attention module (Gsim layer) of the Grassman manifold; it is easy to see that... Each row of elements in the array satisfies convex constraints.

[0188] S2-5-5: As mentioned earlier, the wFM based on PM defines the extrinsic mean on the Grassmann manifold, which leads to GDLNet's inability to effectively utilize and preserve the Riemannian geometric structure of the input data during feature aggregation, thus affecting the system's classification ability. Therefore, this invention utilizes the Orthogonal Group Invariant Riemannian Metric (OGIRM) from an ONB geometric perspective, along with its associated exponential and logarithmic mappings, to calculate the intrinsic mean on the Grassmann manifold.

[0189] By iteratively optimizing, the optimal mean point of a batch of samples can be learned on the Grassmann manifold, exhibiting affine invariance and improving the model's feature representation and classification capabilities. Specifically, this invention designs a network layer called the Affine Invariant Mean (abbreviated as the Aim layer) in IRGANet, and its calculation process is as follows:

[0190] S2-5-5-1: Orthogonal subspace mapping.

[0191] because It represents A q-dimensional subspace in space, where The column vectors are a set of orthogonal basis vectors spanning this subspace. It is easy to see that for any non-singular orthogonal matrix... Depend on The spanned subspace is equivalent to The subspace spanned by this subspace. The space formed by such a set of affinely equivalent sample points is called the affine shape space. In order to calculate the Grassmann mean in this space, this invention first starts from the orthogonal complement space of all-1 vectors (1 ⊥ Choose an orthogonal basis matrix from ) It is obtained through the Gram-Schmidt orthogonalization process. Since affine transformations include operations such as translation, rotation, and scaling, and... The column vectors are orthogonal to the all-one vector, therefore this matrix can be used to eliminate translation effects in the input data. Specifically, using... Each input Project to 1 ⊥ , represented as: Due to E r Each column in the table represents the orthogonal projection of the input data after eliminating the effects of translation. This ensures that the calculation of the Grassmann mean is only related to the relative geometric relationship between the sample points and the shape of the data, and is not affected by displacement factors.

[0192] S2-5-5-2: Mean iterative calculation.

[0193] Next, regarding E j Perform the thin singular value decomposition operation:

[0194]

[0195] Wherein, thinSVD represents the thin singular value decomposition of the matrix. The main purpose of the above decomposition is to extract E j Principal components are used to reduce noise and redundancy, providing meaningful basis vectors for subsequent mean calculation and optimization, thereby improving the algorithm's efficiency and accuracy. Logarithmic mapping and weighted averaging are then applied to... Perform iterative calculations to obtain the results. Specifically:

[0196] Will As the initial value of the mean, i.e. Repeat the following three calculation steps: For ease of explanation, we will use the t-th iteration as an example. In the t-th iteration:

[0197] S2-5-5-2-1: Let the current iteration be the t-th iteration. Use the logarithmic mapping GLog to map the data to the tangent space of the result calculated in the previous iteration. The result is denoted as π. j , π j Specifically:

[0198]

[0199] in, This is the result obtained from the (t-1)th iteration. ρ=tan -1 θ, Here, based on the calculated μθδ T The relationship between ρ and θ is ρ = tan θ -1 θ yields μρδ T The calculation results are obtained from this. The calculation results.

[0200] S2-5-5-2-2: Calculate the arithmetic weighted average of the samples in the tangent space, denoted as τ. The method for calculating τ is as follows:

[0201]

[0202] S2-5-5-2-3: Transform the obtained τ back to the Grassmann manifold using the exponential mapping GExp:

[0203]

[0204] Where, μθδT =thin SVD(τ); here based on μθδ T =thin SVD(τ) is decomposed into μ, θ, δ, and δ, which are then substituted into formula (26) to obtain the calculation result.

[0205] Repeat the iterative calculation process from S2-5-5-2-1 to S2-5-5-2-3 until the iteration stopping condition is met: matrix τ satisfies ‖τ‖ F When the threshold value is less than ε (ε is a very small fixed threshold), or when the maximum number of iterations is reached, the iteration calculation stops, and the result is set to zero. As

[0206] S2-5-5-3: Absolute position projection.

[0207] However, the output in step S2-5-5-2 above This cannot be used as the final optimized Grassman mean because it is the result of decentralized operations. Therefore, the following operation is used to... Projecting back into the original space, the result is denoted as Y′. r ,Y′ r The calculation method is as follows:

[0208]

[0209] because It is 1 ⊥ Given the orthogonal basis matrix in the matrix, it is easy to see that formula (27) ensures the final mean Y′. r It not only contains the geometric structure information of the input data, but also accurately recovers the removed translation components, that is, it recovers the absolute position of the mean point in the original space, thus obtaining the accurate Grassmann mean.

[0210] At this point, one attention calculation process on the Grassmann manifold has been completed, and the resulting new set of subspace data points is represented as follows: This refers to the SPD Grassmann manifold feature. It can be seen that the Gpm layer, Ore layer, OgiRm layer, Gsim layer, and Aim layer together constitute an attention computation module in IRGANet. Compared to GDLNet, IRGANet has advantages in two aspects: 1) It can obtain a unique local optimum using OGIRM and the corresponding GLog and GExp, giving it stronger representational capabilities compared to GDLNet's approximate solution process; 2) Compared to GDLNet's extrinsic mean, the Grassmann mean (intrinsic mean) derived from the Aim layer in IRGANet has affine invariance, making it more suitable for signal classification tasks with deformation characteristics.

[0211] The hybrid manifold modeling module is the core module, designed to explicitly learn the geometric correlations of EEG signals in the spatiotemporal dimensions. This module generates query, key, and value matrices by mapping the input data. For the SPD manifold, a bilinear mapping is used; while for the Grassmann manifold, an orthogonal subspace mapping is employed. Next, the geometric similarity between the data is calculated, using a Riemannian metric to calculate the distance between queries and keys, thereby measuring their correlation.

[0212] S2-6: Although the designed explicit, fine-grained hybrid manifold modeling network can help the SPD manifold self-attention module and the Grassman manifold self-attention module to achieve discriminative statistical encoding and learning of spatiotemporal EEG features, the two failed to perform dynamic interaction of hybrid Riemann features, thus limiting the system's classification ability. Given that feature aggregation (in this invention, this refers to a weighted averaging process on the Riemann manifold) is a core step in the attention framework, and the weight matrix is ​​a key factor in achieving "important information mining and interference information suppression," this invention constructs a Riemann similarity function that integrates LCM and OGIRM to better utilize the complementary spatiotemporal statistical information output by the two Riemann self-attention networks. This function integrates the geodesic distances between Query and Key on different manifolds through weighted summation to obtain more discriminative attention weights, thereby enabling more accurate similarity determination.

[0213] For ease of description, this invention designs a hybrid Riemann similarity metric module (abbreviated as HRsim layer) between LCMANet and IRGANet. The hybrid Riemann similarity metric module integrates geodesic distances on different manifolds through weighted summation, and the calculation formula is as follows:

[0214]

[0215] in The values ​​represent the attention weight coefficients output by the hybrid Riemann similarity metric module, r = 1, ..., s, j = 1, ..., s. Formulas (11) and (21) respectively give and The specific mathematical expression, ξ1 and ξ2 are equilibrium parameters; ξ1 and ξ2 (learnable) are used to eliminate and The difference lies in the magnitude, and on the other hand, it is used to dynamically adjust the information of each branch. The proportion in. One point needs to be added: due to... What is being measured is the angle between subspaces, therefore it needs to be multiplied by . To convert it to radians, thus preserving and Consistency. At this point, use... This represents the attention weight matrix output by the HRsim layer.

[0216] Finally, let This enables dynamic feature interaction based on the attention matrix.

[0217] S2-7: The Feature Fusion-Based EEG Signal Classification Module (FFCM) classifies EEG signals. To achieve the EEG signal classification task, the output data of LCMANet and IRGANet are first mapped to Euclidean space, i.e., the tangent space at the identity matrix on the manifold, using the logarithmic mapping function of the matrix. Specifically:

[0218] A manifold-Euclidean embedding mapping layer, denoted as SMEem, is constructed at the output of the SPD manifold self-attention module LCMANet, and a manifold-Euclidean embedding mapping layer, denoted as GMEem, is constructed at the output of the Grassmann manifold self-attention module IRGANet. The calculation formula for the SMEem layer is as follows, using the logarithmic mapping function induced by LCM:

[0219]

[0220] in, Let r denote a symmetric matrix located in Euclidean space, r = 1 → s.

[0221] Similarly, using the logarithmic mapping function shown in formula (24), the calculation formula for the GMEem layer is as follows:

[0222]

[0223] in, This indicates the truncated identity matrix. Let r denote a column full-rank matrix in Euclidean space, where r = 1 → s.

[0224] Next, this invention constructs a hybrid feature fusion layer (abbreviated as Hff layer) in FFCM, which mainly outputs Euclidean feature vectors, denoted as ψ, through feature concatenation. r , ψ r The calculation formula is:

[0225]

[0226] Where r = 1 → s, Indicates taking out The lower triangular portion is then stretched into a column vector. Indicates will To stretch the vector into column vectors, `Concat(,)` refers to concatenating features along the column direction. (d′ t =dt ×(d t +1) / 2+d t ×q) is the Euclidean eigenvector output by the Hff layer.

[0227] Finally, a submodule including the FC layer and the Softmax function is embedded at the end of the hybrid feature fusion layer to achieve signal classification.

[0228] The specific implementation process of the hybrid manifold attention network model is as follows: Figure 4 As shown.

[0229] S3: Train the hybrid manifold attention network model using the training set, and input the test set into the trained hybrid manifold attention network model to obtain the EEG signal classification results.

[0230] Existing models primarily rely on Euclidean neural networks, such as CNNs and Transformers, for nonlinear representation learning mechanisms, neglecting the inherent low-dimensional submanifold structure of sequence data like EEG. Effectively representing the submanifold structure of EEG data is a major factor influencing the development of brain-computer interface technology. Considering that the Grassmann manifold is a subspace manifold, it is essentially isomorphic to the inherent low-dimensional submanifold structure of EEG data. Furthermore, Euclidean attention networks have been proven to effectively encode long-range correlations between data. Therefore, by leveraging Riemannian geometric operators from the perspective of orthonormal basis (ONB) on the Grassmann manifold, this invention extracts complementary Riemannian geometric representations through a manifold modeling module. Compared to existing methods that use extrinsic means based on projection metric (PM) (an approximate Grassmann manifold mean calculation strategy), the improved model, utilizing intrinsic means from the ONB perspective, more effectively preserves the Riemannian geometric structure of the input data, thereby enhancing its classification ability for EEG data.

[0231] While many methods have attempted to represent the non-Euclidean structure of EEG data using Riemannian manifolds, they typically rely on Riemannian descriptors, such as the covariance matrix, to map the entire EEG signal sequence to a single sample point on the manifold. This is essentially an implicit, coarse-grained EEG representation mechanism, failing to explicitly mine and utilize the spatiotemporal correlations of EEG signals, thus affecting classification accuracy. To address this issue, this invention first utilizes a simple convolutional module (containing only two convolutional layers) to extract the spatiotemporal semantic features of EEG signals. Next, the data is simultaneously segmented along both the temporal and channel dimensions, yielding a set of data (Tokens) containing spatiotemporal attributes. Based on this, a hybrid manifold modeling module implements explicit, fine-grained spatiotemporal cross-attention encoding and learning between Tokens. By combining the feature extraction module, manifold modeling module, and hybrid manifold modeling module, the geometric correlations of EEG signals in the spatiotemporal dimensions are explicitly and finely mined and encoded.

[0232] Most Riemannian manifold-based EEG classification algorithms are typically built on a single SPD manifold, failing to fully explore and utilize the statistical complementarity between heterogeneous manifolds. Effectively integrating and utilizing the advantages of different matrix manifolds to achieve more robust and efficient EEG data decoding remains a key technical challenge. To address this issue, this invention proposes simultaneously modeling the obtained tokens onto both Grassmann and SPD manifolds to obtain complementary Riemannian geometric features. Then, a designed hybrid manifold attention network is used to achieve spatiotemporal encoding, discriminative learning, and fusion classification of the data. Furthermore, a similarity function based on a hybrid Riemannian metric is designed, using a hybrid Riemannian similarity metric module to accurately determine the geometric similarity between manifold data, thereby improving the model's learning ability for EEG signals.

[0233] Existing SPD manifold attention networks are built upon the Log-Euclidean Metric (LEM) and its associated exponential, logarithmic, and Riemannian means. Because their computation involves multiple matrix eigenvalue operations (which GPUs cannot accelerate), the inference speed of these models is slow, making it difficult to meet the low-latency requirements of practical applications. To address this issue, this invention proposes LCMANet, an SPD manifold self-attention network based on the Log-Cholesky Metric (LCM). Compared to LEM, Riemannian operators based on LCM, such as logarithmic, exponential, and Riemannian means, do not involve any eigenvalue operations (the inverse operation only applies to diagonal elements) and all have closed-form solutions. Furthermore, the attention computation process under LCM involves only one Cholesky decomposition of the matrix. The aforementioned characteristics enable the proposed model to outperform existing models in terms of computational efficiency. Furthermore, compared to the quadratic dependency of LEM on the SPD matrix, the quadratic dependency of LCM only applies to the main diagonal elements (quadratic dependency makes the metric framework more sensitive to ill-conditioned matrices). The SPD manifold self-attention module is built based on the log-Cholsky metric, which improves the robustness and computational efficiency of the model, enabling it to meet the stability requirements of practical applications.

[0234] This invention also discloses an EEG signal classification system based on a hybrid manifold attention network, comprising: a data acquisition module for acquiring EEG signals and dividing them into training and testing sets; and a hybrid manifold attention network model construction module for constructing a hybrid manifold attention network model, wherein the hybrid manifold attention network model includes a feature extraction module, a manifold modeling module, and a hybrid manifold modeling module. The feature extraction module extracts the spatiotemporal semantic features of the EEG signals; the manifold modeling module divides the spatiotemporal semantic features of the EEG signals in both time and channel dimensions to obtain spatiotemporal token data and space-time token data, and simultaneously models the spatiotemporal token data and space-time token data onto both the SPD manifold and the Grassmann manifold. The hybrid manifold modeling module includes an SPD manifold self-attention module, a Grassmann manifold self-attention module, and a hybrid Riemann similarity metric module. The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollesky metric, the Grassmann manifold self-attention module extracts Grassmann manifold features, and the hybrid Riemann similarity metric module integrates geodesic distances on different manifolds through weighted summation. A training module is used to train the hybrid manifold attention network model using a training set. A classification module is used to input the test set into the trained hybrid manifold attention network model to obtain EEG signal classification results.

[0235] This invention designs a lightweight geometric deep learning framework. Its core component is a hybrid manifold modeling module, used to explicitly and finely encode the geometric correlations of EEG spatiotemporal features and achieve discriminative learning. Furthermore, it includes a convolution module, a manifold modeling module, and a classification module, used to extract spatiotemporal-space-time features of EEG signals, simultaneously map these features to SPD and Grassmann manifolds to achieve complementary representations, and perform pattern classification, respectively. Compared with existing technologies, the advantages of this invention are as follows:

[0236] 1. Enhance the robustness of EEG data modeling

[0237] While numerous methods have attempted to model the geometric structure of EEG data using Riemannian manifolds, they typically map the entire EEG signal sequence to a single element on the manifold. This approach is inherently an implicit, coarse-grained EEG representation mechanism, failing to explicitly mine and utilize the spatiotemporal correlations of EEG signals. Furthermore, most Riemannian manifold-based EEG classification algorithms are usually built within a single SPD manifold framework, failing to fully exploit the complementarity of heterogeneous manifolds in Riemannian geometry. This invention utilizes a constructed hybrid manifold modeling network to not only achieve joint mining of spatiotemporal and spatiotemporal semantic information from EEG signal data but also to achieve statistically complementary representations of the intrinsic submanifold structures of the data. This explicit, fine-grained spatiotemporal statistical representation mechanism provides more robust input features for subsequent attention calculations, enhancing the model's ability to learn EEG signal representations.

[0238] 2. Improve the real-time performance of the EEG classification system:

[0239] The original SPD manifold attention network, Matt, is built upon LEM and its associated exponential, logarithmic, and Riemannian means. Because its computation involves multiple matrix eigenvalue operations (which GPUs cannot accelerate), the model's inference speed is slow, making it difficult to meet the low-latency requirements of practical applications. This invention utilizes the designed LCMANet to effectively improve the model's computational speed, with advantages in two aspects. First, LCMANet uses Cholesky decomposition and its inverse mapping to equivalently transfer the attention computation process on the SPD manifold to the Cholesky space. Its Euclidean space embedding submanifold structure allows the proposed attention model to more effectively encode and learn the EEG data structure, improving the representation learning capability of the classification system. Second, thanks to LCM, LCMANet's computation process does not involve complex matrix functions (such as SVD), requiring only one Cholesky decomposition and its inverse mapping, significantly improving the computational efficiency of the EEG classification system.

[0240] 3. Enhance the ability to learn representations from EEG data.

[0241] Existing models typically ignore the inherent low-dimensional submanifold structure of sequence data like EEG. Effectively representing this submanifold structure is a major factor influencing the development of brain-computer interface technology. Given that the Grassman manifold is a subspace manifold (essentially isomorphic to the inherent low-dimensional submanifold structure of EEG data), this invention improves and optimizes the original GDLNet, proposing an enhanced Grassman manifold attention network, IRGANet. This network utilizes OGIRM and associated logarithmic and exponential mappings to obtain a unique local optimum for the Riemann mean. Compared to the approximate solution process of GDLNet, IRGANet exhibits stronger representation learning capabilities. Furthermore, compared to the extrinsic Riemann mean in GDLNet, the Riemann mean (intrinsic mean) derived by IRGANet possesses affine invariance, making it more suitable for EEG signal classification tasks with complex deformation characteristics.

[0242] 4. Enhance the discriminative learning ability of EEG classification models.

[0243] Given that the weighted averaging process (feature aggregation) on the Riemannian manifold is a core step in the attention framework, and the weight matrix is ​​a key factor in achieving "important information mining and interference information suppression," this invention optimizes the feature aggregation process in the manifold attention network and designs a Riemann similarity determination function that integrates LCM and OGIRM. This function generates shareable weight parameters for the designed IRGANet and LCMANet, enabling dynamic interaction of hybrid Riemann features. Compared to similarity calculation functions that integrate a single Riemannian metric and feature, the attention weights obtained by the above method are more discriminative, thereby enhancing the model's ability to classify EEG signals and improving its adaptability in complex scenarios.

[0244] This invention effectively addresses the shortcomings of existing technologies in EEG data modeling robustness, real-time classification, characterization discriminative power, and scene adaptability. It significantly improves the accuracy and real-time performance of EEG signal classification, enhances the system's adaptability and robustness, and reduces the error rate of EEG signal decoding. It can better handle EEG signal data with complex spatiotemporal characteristics while meeting low-latency requirements. These beneficial effects not only meet the demands for efficient and high-precision classification but also promote the application of Riemannian geometry-based EEG decoding technology in real-world scenarios, demonstrating broad practical application value.

[0245] To further demonstrate the advantages of this invention, two representative EEG datasets were selected for classification experiments in this embodiment. The two representative EEG datasets are: the MAMEM-SSVEP-II dataset and the BCI-ERN dataset.

[0246] The first step is to prepare and preprocess the dataset. First, noise reduction is performed by using filtering techniques to remove noise outside the frequency range of the EEG signal. Specifically, for the MAMEM-SSVEP-II dataset, the preprocessing steps include: (1) performing bandpass filtering from 1 to 50 Hz, (2) selecting eight channels (PO7, PO3, PO, PO4, PO8, O1, Oz, and O2) related to the occipital region (visual cortex), and (3) dividing each trial into four 1-second segments within a time period of 1 to 5 seconds after the stimulus cue begins, thereby generating 500 8-channel SSVEP signal trials with a length of 1 second for each subject. For the BCI-ERN dataset, the preprocessing steps include: (1) adjusting the sampling rate of the data recorded in 340 trials from 600 Hz to 128 Hz, and (2) performing bandpass filtering from 1 to 40 Hz to finally obtain the preprocessed ERN signal samples.

[0247] The experiment employed a cross-session validation approach to ensure fairness in model training and evaluation. The dataset contained five sessions, specifically divided as follows: the first three sessions (sessions 1, 2, and 3) were used for training, session 4 was used as the validation set, and session 5 was used as the test set. This division aimed to maximize the effectiveness of cross-session validation, thereby improving the model's generalization ability.

[0248] The second step is model structure setup and training.

[0249] Feature extraction module settings: For the MAMEM-SSVEP-II dataset, the kernel size of the first convolutional layer is (8,1), and the number of input / output feature channels is 1 / 125; the kernel size of the second convolutional layer is (1,36), and the number of input / output feature channels is 125 / 15; for the BCI-ERN dataset, the kernel size of the first convolutional layer is (56,1), and the number of input / output feature channels is 1 / 22; the kernel size of the second convolutional layer is (1,64), and the number of input / output feature channels is 22 / 16.

[0250] Configure the hybrid manifold modeling module: For the MAMEM-SSVEP-II dataset, the sizes of the SPD matrix and the orthogonal basis matrix are configured as 15×15 and 15×8, respectively; for the BCI-ERN dataset, the sizes of the SPD matrix and the orthogonal basis matrix are configured as 15×15 and 16×8, respectively.

[0251] In the computation of the hybrid manifold modeling module, two learnable equilibrium parameters, ξ1 and ξ2, are introduced to eliminate the dimensional differences in distance between the LCM and OGIRM associated with the SPD and Grassmann manifolds. Their initial values ​​are both set to 1. Based on the geodesic distances of the hybrid manifolds, an attention matrix is ​​constructed and Softmax normalization is performed. Finally, feature fusion is performed on the SPD and Grassmann manifolds using LCM-based and OGIRM-based weighted Fréchet means, respectively, outputting weighted aggregated manifold features. Ultimately, the size of the feature matrix output by HMANet on different manifolds is as follows: 8×8 for the SPD manifold and 12×8 for the Grassmann manifold. Finally, EEG signal classification is performed using the feature fusion-based classification module FFCM. This process first concatenates the hybrid manifold features output by HMANet in the tangent space. Then, by using a submodule consisting of FC and Softmax, the input features are mapped to the category hypersphere to achieve the classification task of EEG signals.

[0252] On two EEG datasets, the present invention achieved classification results superior to existing models, verifying its effectiveness and advancement.

[0253] In the field of brain-computer interfaces (BCI), this invention can be applied to real-time pattern analysis of EEG signals. For example, in the visual stimulation (SSVEP) task, it can identify the subject's EEG response in real time, driving a cursor or device to point to different locations. In the medical field, HMANet can be used to monitor abnormal patterns in EEG signals, such as predicting epileptic seizures or other neurological diseases. By performing real-time, robust feature modeling, representation learning, and pattern classification on EEG signals, abnormalities can be detected promptly and effective interventions can be implemented.

[0254] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A method for classifying electroencephalogram (EEG) signals based on a hybrid manifold attention network, characterized in that, include: EEG signals were acquired and divided into training and testing sets. A hybrid manifold attention network model is constructed, comprising a feature extraction module, a manifold modeling module, and a hybrid manifold modeling module. The feature extraction module extracts the spatiotemporal semantic features of the EEG signal. The manifold modeling module divides the spatiotemporal semantic features of the EEG signal into spatiotemporal token data and space-time token data in both time and channel dimensions, and models the spatiotemporal token data and space-time token data onto both the SPD manifold and the Grassmann manifold simultaneously. The hybrid manifold modeling module includes an SPD manifold self-attention module, a Grassman manifold self-attention module, and a hybrid Riemann similarity measurement module. The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, the Grassman manifold self-attention module extracts Grassman manifold features, and the hybrid Riemann similarity measurement module integrates geodesic distances on different manifolds through weighted summation. The hybrid manifold attention network model is trained using the training set, and the test set is input into the trained hybrid manifold attention network model to obtain the EEG signal classification results.

2. The EEG signal classification method based on a hybrid manifold attention network according to claim 1, characterized in that: The process of dividing the spatiotemporal semantic features of EEG signals along both time and channel dimensions to obtain spatiotemporal token data and spatiotemporal token data is as follows: The spatiotemporal semantic features of the EEG signal are denoted as feature matrix E. i E i The size is c×l, for the characteristic matrix E i Fine-grained segmentation is performed along both the channel and time dimensions: In the spatial dimension, E i The data is divided into m continuous and non-overlapping segments, each segment having a size of h×l, where h = c / m; the resulting space-time token data is used... express, This represents the j-th empty time token data, where j = 1, ..., m; In the time dimension, E i Divide the data into n consecutive and non-overlapping segments, each segment having a size of c × d, where d = l / n; This indicates the obtained spatiotemporal token data. This represents the k-th spatiotemporal token data, where k = 1, ..., n.

3. The EEG signal classification method based on a hybrid manifold attention network according to claim 2, characterized in that: The process of simultaneously modeling spatiotemporal token data and spacetime token data onto SPD manifolds and Grassman manifolds specifically involves: The SPD manifold representation corresponding to the spatiotemporal token data and the spatiotemporal token data is as follows: For the j-th space-time token data, the formula for calculating the covariance is: in, express The covariance characterization, express The eigenvector of the t-th column, t = 1, ..., l, express The column mean; For the k-th spatiotemporal token data, the formula for calculating the covariance is: in, express The covariance characterization, express The eigenvectors in the r-th row of the array, where r = 1, ..., c, express The row mean; Use regularization techniques to ensure and All are SPD matrices: Where I represents the identity matrix, Trace(·) refers to the trace of the matrix, and λ is a perturbation constant; The Grassman manifold representations corresponding to spatiotemporal token data and spacetime token data are obtained by using the singular value decomposition of the matrix, specifically: The j-th spacetime token data and k spatiotemporal token data The corresponding subspace is represented as: in, They represent The corresponding subspace representation, They represent The eigenvector matrix corresponding to the first q largest eigenvalues, and This is the corresponding eigenvalue matrix, where T represents the transpose; At this moment, the original feature matrix E i The corresponding hybrid manifold representations are expressed in the following set language as follows: in, and They refer to E respectively i The corresponding SPD manifold representation and Grassmann manifold representation, due to and and All have the same size, and for ease of subsequent description, the following simplified data representation form is adopted: Where s = m + n.

4. The EEG signal classification method based on a hybrid manifold attention network according to claim 3, characterized in that: The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, specifically: The query Q, key K, and value V on the SPD manifold are generated using a bilinear mapping, as shown below: Among them, Q r K r V r Let Q, K, and V represent the feature matrices corresponding to the r-th query Q, key K, and value V on the SPD manifold, respectively. (d t ≤d) are three projection matrices that satisfy column orthogonality; Q is obtained using Cholesky decomposition. r K r V r The corresponding query in the Choreskiy manifold key Sum Recorded as The calculation formula is as follows: in, These are the three lower triangular matrices obtained; Using LCM calculation and The geodesic distance between them is calculated using the following formula: in, for and Geodesic distances between them, r = 1, ..., s, j = 1, ..., s, Reference A strictly lower triangular matrix. Indicates by The diagonal matrix formed by the main diagonal; Log represents the logarithm of the matrix. calculate The attention weight coefficient is calculated using the following formula: in, for Attention weight coefficients; This represents the attention weight matrix output by the self-attention module of the SPD manifold; The weighted Frescher mean is used to aggregate features in a manifold, and the calculation formula is as follows: in, For feature aggregation in a manifold, Representing the Choreskiy manifold, Referring to the Riemannian metric on the Choreskiy manifold, This represents the mean of the manifold that needs to be optimized. The closed-form solution is shown below: For the solution The inverse operation of the Cholliski decomposition is used to map it back to the SPD manifold, and the calculation formula is as follows: Among them, C′ r for The representation after mapping back to the SPD manifold; At this point, one attention calculation process on the SPD manifold has been completed, and the resulting new set of SPD data points is represented as follows: That is, the SPD manifold characteristics.

5. The EEG signal classification method based on a hybrid manifold attention network according to claim 4, characterized in that: The Grassmann manifold self-attention module extracts Grassmann manifold features, specifically as follows: for The query Q, key K, and value V on the Grassman manifold are calculated using the following formula: in, Let Q, K, and V represent the feature matrices corresponding to the r-th query Q, key K, and value V on the Grassman manifold, respectively. (d t ≤d) are three projection matrices that satisfy column orthogonality; right Perform the QR decomposition as shown below: in, It is an orthogonal matrix. It is an upper triangular matrix; use the following formula to... The transformation into an eigenma matrix with column orthogonality is shown below: in, for The transformed feature matrix with column orthogonality; Use and Calculation Obtained in the same way and Calculation using OGIRM and The geodesic distance between them is calculated using the following formula: in, for and Geodesic distances between them, r = 1, ..., s, j = 1, ..., s; ∑ a Let q be the a-th main diagonal element in ∑, where a = 1, ..., q; thinSVD represents the thin singular value decomposition of a matrix, i.e. θ a Let Θ be the a-th main diagonal element, and Θ = cos -1 ∑; Will The formula for converting to attention weight coefficients is shown below: Where, η rj for Attention weight coefficients, using This represents the attention weight matrix output by the self-attention module of the Grassman manifold; Using the orthogonal group invariant Riemannian metric OGIRM from the perspective of ONB geometry, and its associated exponential and logarithmic mappings, we calculate the intrinsic mean on the Grassmann manifold. At this point, one attention calculation process on the Grassmann manifold has been completed, and the resulting new set of subspace data points is represented as follows: That is, the SPD Grassmann manifold feature.

6. The EEG signal classification method based on a hybrid manifold attention network according to claim 5, characterized in that: The intrinsic mean on the Grassmann manifold is calculated using the orthogonal group invariant Riemannian metric OGIRM from the ONB geometric perspective, along with its associated exponential and logarithmic mappings. The calculation process is as follows: Choose an orthogonal basis matrix from the orthogonal complement space of all-one vectors. use Each input Project to 1 ⊥ , is represented as: For E j Perform the thin singular value decomposition operation: Wherein, thinSVD represents the thin singular value decomposition of the matrix. Logarithmic mapping and weighted average will be used to... Perform iterative calculations to obtain the results. Will Projecting back into the original space, the result is denoted as Y′. r ,Y′ r The calculation method is as follows:

7. The EEG signal classification method based on a hybrid manifold attention network according to claim 6, characterized in that: The method of logarithmic mapping and weighted averaging will be used to... Perform iterative calculations, specifically: Will As the initial value for the mean, repeat the following steps: Let the current iteration be the t-th iteration. Using the logarithmic mapping GLog, we map the data to the tangent space of the result calculated in the previous iteration, denoted as π. j , π j Specifically: in, This is the result obtained from the (t-1)th iteration. ρ=tan -1 θ, Calculate the arithmetic weighted average of the samples in the tangent space, denoted as τ. The method for calculating τ is as follows: The obtained τ is transformed back to the Grassmann manifold using the exponential mapping GExp: among which T =thin SVD(τ); Repeat the above iterative calculation process until the iteration stopping condition is met, and then stop the iteration calculation. As 8. The EEG signal classification method based on a hybrid manifold attention network according to claim 5, characterized in that: The hybrid Riemann similarity metric module integrates geodesic distances on different manifolds through weighted summation, and the calculation formula is as follows: in The values ​​represent the attention weight coefficients output by the hybrid Riemann similarity metric module, r = 1, ..., s, j = 1, ..., s. ξ1 and ξ2 are equilibrium parameters; make Implement dynamic feature interaction based on attention matrix.

9. The EEG signal classification method based on a hybrid manifold attention network according to claim 7, characterized in that: The hybrid manifold attention network model also includes a signal classification module, specifically: An embedded mapping layer, denoted as SMEem, is constructed at the output of the SPD manifold self-attention module, and an embedded mapping layer, denoted as GMEem, is constructed at the output of the Grassmann manifold self-attention module. Using the logarithmic mapping function induced by LCM, the SMEem layer is calculated as follows: in, Denotes a symmetric matrix located in Euclidean space; The calculation formula for the GMEem layer is as follows: in, This indicates the truncated identity matrix. Represents a column-full-rank matrix in Euclidean space; Construct a hybrid feature fusion layer, and output a Euclidean feature vector, denoted as ψ, through feature concatenation. r , ψ r The calculation formula is: in, Indicates taking out The lower triangular portion is then stretched into a column vector. Indicates will To stretch the vector into column vectors, `Concat()` refers to concatenating features along the column direction. The FC layer and Softmax function are embedded at the end of the hybrid feature fusion layer to achieve EEG signal classification.

10. A classification system for electroencephalogram (EEG) signals based on a hybrid manifold attention network, characterized in that, include: The data acquisition module is used to acquire EEG signals and divide them into training and test sets. A hybrid manifold attention network model construction module is used to construct a hybrid manifold attention network model. The hybrid manifold attention network model includes a feature extraction module, a manifold modeling module, and a hybrid manifold modeling module. The feature extraction module extracts the spatiotemporal semantic features of the EEG signal. The manifold modeling module divides the spatiotemporal semantic features of the EEG signal in both time and channel dimensions to obtain spatiotemporal token data and space-time token data, and simultaneously models the spatiotemporal token data and space-time token data onto the SPD manifold and the Grassmann manifold. The hybrid manifold modeling module includes an SPD manifold self-attention module, a Grassman manifold self-attention module, and a hybrid Riemann similarity measurement module. The SPD manifold self-attention module extracts SPD manifold features based on the log-Chollisky metric, the Grassman manifold self-attention module extracts Grassman manifold features, and the hybrid Riemann similarity measurement module integrates geodesic distances on different manifolds through weighted summation. The training module is used to train the hybrid manifold attention network model using the training set; The classification module is used to input the test set into the trained hybrid manifold attention network model to obtain the classification results of the EEG signals.

Images