Electroencephalogram data action intention decoding method and system based on multi-modal clustering

Abstract

The application discloses a brain electrical data motion intention decoding method and system based on multi-modal clustering, and the method receives multi-channel brain electrical signals, performs frequency filtering and feature extraction in accordance with physiological rhythms, and constructs a multi-modal feature set; then, a consensus graph is decomposed by a clustering model to learn a global consistent structure and local specific structures of each view, that is, a low-rank decomposition technology is used to extract global information, and a projection matrix with constraints is introduced to deeply mine local manifold structures of each view; finally, an optimization solution is used to output a decoding result. The application provides a more accurate and stable control instruction generation scheme for a brain-computer interface system.

Description

Technical Field

[0001] This invention relates to the fields of signal processing and data analysis technology, and in particular to a method and system for decoding action intentions from electroencephalogram (EEG) data based on multimodal clustering. Background Technology

[0002] Brain-computer interfaces (BMIs) can directly translate electrical, magnetic, or metabolic brain signals into control commands for external devices such as robots, prosthetics, or exoskeletons. With advancements in acquisition technology, the dimensionality and scale of EEG multiview data continue to grow, and obtaining precise labels (such as specific motor intentions) for this data typically requires significant manpower and time. Therefore, unsupervised learning methods that do not rely on labels, especially multiview clustering techniques, are becoming increasingly important in EEG decoding. Multi-View Subspace Clustering (MVSC), as a mainstream method, aims to effectively cluster multiview data by learning a subspace representation for each view and constructing a shared consensus graph to explore the consistency structure among all views.

[0003] However, existing multi-view subspace clustering methods still have significant shortcomings when applied to EEG decoding scenarios. First, most methods focus on constructing a unified consensus graph to integrate multi-view information, but this process easily overlooks the unique and discriminative local structural information of each view (such as different frequency bands or feature domains). This neglect of view differences may result in the finally learned shared representation only reflecting the characteristics of the view with the richest information, failing to fully utilize the complementary information of all views, thus limiting the discriminative power and decoding accuracy of the clustering model. Second, existing methods usually learn the consensus graph as a whole, lacking explicit modeling and differentiation of its internal components, making it difficult to finely balance the global consistency between views and the local specificity of each view, thus limiting the model's ability to express complex EEG pattern relationships. In addition, the first-order consensus graph constructed by traditional methods (considering only the direct adjacency relationship between samples) often suffers from sparse connections and incomplete structural information when processing high-noise, low-signal-to-noise ratio EEG data, making it difficult to capture deeper and more complex nonlinear relationships between samples, thus restricting the model's ability to discover robust underlying structures from multi-view EEG data.

[0004] Therefore, there is an urgent need for a high-performance clustering method that can effectively integrate the global consistency structure of EEG multi-view data with the local specific information of each view, and can mine deep relationships in the data, so as to improve the accuracy and robustness of EEG decoding. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for decoding action intent from EEG data based on consensus graph decomposition and high-order graph fusion multimodal subspace clustering. The consensus graph decomposition is divided into a base graph responsible for extracting a globally consistent structure from all views and a special graph set that preserves the unique local adjacency information of each view. High-order graph technology is introduced to establish multivariate associations between samples and capture the hidden high-order correlations between views. This can effectively improve the accuracy of multi-view subspace clustering of EEG data and significantly improve the clustering efficiency. Thus, action intent decoding instructions for controlling external devices are generated based on the action intents corresponding to the subspace clustering results.

[0006] The technical solution adopted in this invention is:

[0007] A method for decoding action intent from EEG data based on multimodal clustering includes the following steps:

[0008] Acquire multi-channel EEG data, which includes raw EEG data from multiple leads and electrooculography (EOG) signals from at least one lead;

[0009] Frequency band filtering of EEG data yields multiple frequency band signals corresponding to different EEG physiological rhythms;

[0010] Temporal and frequency domain features are extracted for each frequency band signal to construct a multimodal EEG feature dataset containing multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view;

[0011] Multimodal subspace clustering is performed based on a multimodal EEG feature dataset to identify EEG patterns representing different motor intentions, resulting in subspace clustering results distinguished by EEG physical characteristics.

[0012] Generate motion intent decoding instructions for controlling external devices based on the motion intents corresponding to the subspace clustering results;

[0013] Among them, multimodal subspace clustering uses low-rank decomposition to extract global information and introduces a constrained projection matrix to deeply explore the local manifold structure of each view. The subspace clustering results are obtained by solving the following optimization problem:

[0014]

[0015] Where S is the consensus graph set, , It is the feature matrix of the v-th view. consensus graph matrix, This represents the k-th order graph of the v-th view. To select a matrix, Indicates corresponding to Boolean value; Base map, For special diagrams; , , For regularization parameters; It is the identity matrix; Represents graph embedding; For Laplace matrix, It is a degree matrix; It is a matrix whose elements are all 1s.

[0016] Furthermore, the implementation of acquiring multi-channel EEG data includes:

[0017] The TGAM brainwave sensor acquires the potential signals of neurons in the scalp to form bioelectrical information, which is then transmitted to the processing device and converted into simulated electroencephalogram (EEG) data.

[0018] The simulated EEG data is filtered out by pre-filtering to remove DC drift and UHF noise, resulting in the original EEG timing matrix as the original EEG data, while retaining the true physical amplitude and phase information.

[0019] The electrooculogram (EOG) signals from the leads of the raw EEG data are acquired and combined with the raw EEG data to form multichannel EEG data.

[0020] Furthermore, the frequency band filtering includes filtering the raw EEG data using bandpass filters corresponding to Delta, Theta, Alpha, Beta, and Gamma EEG rhythms, respectively; the filtering bandwidth for Delta is 0.5~4 Hz; the filtering bandwidth for Theta is 4~8 Hz; the filtering bandwidth for Alpha is 8~13 Hz; the filtering bandwidth for Beta is 13~30 Hz; and the filtering bandwidth for Gamma is 30~50 Hz.

[0021] Furthermore, the time-domain characteristics include at least one of the following: mean, standard deviation, root mean square, skewness, kurtosis, activity, mobility, and complexity; the frequency-domain characteristics include at least one of the following: absolute power, relative power, and spectral centroid of each physiological rhythm frequency band.

[0022] Furthermore, multimodal subspace clustering iteratively updates the variables in the optimization problem using the alternating direction multiplier method. The process continues until the convergence condition is met to obtain the final subspace representation; based on the final subspace representation, the Laplace rank constraint method is applied to obtain the final clustering result.

[0023] Furthermore, the implementation of obtaining the final subspace representation by iteratively updating the optimization problem using the alternating direction multiplier method includes:

[0024] S1, Initialize all variables Set the maximum number of iterations. ;

[0025] S2, corresponding to the Lagrange function is as follows:

[0026] ;

[0027] in, , It is the regularization coefficient, and X is a multimodal EEG feature dataset with multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view It is a Lagrange multiplier. To find the trace of the corresponding matrix;

[0028] S3, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ;

[0029] S4, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ;

[0030] S5, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ;

[0031] S6, Update with other variables fixed Its subproblems are: ;

[0032] S7, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ;

[0033] S8, update while keeping other variables fixed. The update expression is ; ; For the updated parameters ;

[0034] S9, Repeat the parameter update steps until convergence or Stop at this point; obtain the final subspace representation. .

[0035] Furthermore, the present invention also discloses a brainwave data action intention decoding system based on multimodal clustering, which applies the aforementioned brainwave data action intention decoding method based on multimodal clustering. The system includes the following steps:

[0036] The data acquisition module is used to acquire multi-channel EEG data, which includes raw EEG data from multiple leads and electrooculography (EOG) signals from at least one lead.

[0037] The frequency band filtering module, connected to the data acquisition module, is used to perform frequency band filtering on EEG data to obtain multiple frequency band signals corresponding to different EEG physiological rhythms.

[0038] The feature extraction module is used to extract time-domain and frequency-domain features for each frequency band signal to construct a multimodal EEG feature dataset containing multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view;

[0039] The subspace clustering module is used to perform multimodal subspace clustering based on a multimodal EEG feature dataset to identify EEG patterns representing different action intentions and obtain subspace clustering results distinguished by EEG physical characteristics.

[0040] The intent decoding module is used to generate action intent decoding commands for controlling external devices based on the action intents corresponding to the subspace clustering results.

[0041] Furthermore, the data acquisition module is connected to the TGAM brainwave sensor, which acquires bioelectrical information formed by the potential signals of scalp neurons. The data acquisition module converts the bioelectrical information into analog EEG data and then filters out DC drift and ultra-high frequency noise through pre-filtering to obtain the original EEG timing matrix as the original EEG data. The data acquisition module summarizes the original EEG data with the corresponding lead electrooculogram signals to form multi-channel EEG data.

[0042] The present invention adopts the above technical solution and has the following beneficial effects compared with the prior art:

[0043] (1) In view of the technical defects of existing methods that tend to ignore the unique information of each view (such as different frequency bands and different feature domains) when fusing multi-view EEG data, resulting in the loss of effective discrimination features, this invention explicitly decomposes the consensus graph into a base graph and a special graph, and introduces a projection matrix specifically used to mine the local manifold structure of each view, which can make more refined and complete use of the complementary information of all views.

[0044] (2) The inherent low signal-to-noise ratio and individual variability of EEG data require the decoding model to have good robustness. Existing methods based on a single consensus graph are not adequately adapted to this. This invention effectively suppresses interference from abnormal noise or artifacts in the signal by constraining the global structural consistency through low-rank decomposition and the sparsity constraint of the projection matrix, and learns a more fundamental EEG pattern representation.

[0045] (3) This invention decomposes the consensus graph into a base graph and special graphs, whereby the base graph is used to learn the consistency structure among the views, while the special graphs are used to capture the local information unique to each view. By introducing corresponding norm constraints, the consensus structure of the base graph is strengthened, and the size of the special graph is controlled to retain unique information.

[0046] This invention achieves a substantial improvement in the accuracy of decoding motor intentions from EEG data, a significant enhancement in system robustness, and a full guarantee of the feasibility of technology implementation, thereby promoting the overall development of high-performance, high-reliability brain-computer interface systems. Attached Figure Description

[0047] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments;

[0048] Figure 1 This is a flowchart illustrating the action intent decoding method for EEG data based on multimodal clustering according to the present invention.

[0049] Figure 2 This is a schematic diagram of the multi-view clustering framework based on consensus graph decomposition and high-order graph fusion of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0051] like Figure 1 or Figure 2 As shown, this invention discloses a method for decoding action intentions from EEG data based on multimodal subspace clustering fused with consensus graphs and higher-order graphs, comprising the following steps:

[0052] Multi-channel EEG data is acquired, including raw EEG data from multiple leads and electrooculogram (EOG) signals from at least one lead. The acquisition of multi-channel EEG data further includes: acquiring scalp neuronal potential signals using a TGAM brainwave sensor to form bioelectrical information, and transmitting this information to a processing device via Bluetooth serial protocol to convert it into analog EEG data; the analog EEG data is pre-filtered to remove DC drift and UHF noise to obtain the raw EEG timing matrix as the raw EEG data, retaining the true physical amplitude and phase information; and the EOG signals from the leads of the raw EEG data are acquired and combined with the raw EEG data to form multi-channel EEG data.

[0053] The EEG data is subjected to frequency band filtering to obtain multiple frequency band signals corresponding to different EEG physiological rhythms. Further, the frequency band filtering includes filtering the raw EEG data with bandpass filters corresponding to Delta, Theta, Alpha, Beta, and Gamma EEG rhythms respectively. The filtering bandwidth of Delta is 0.5~4 Hz; the filtering bandwidth of Theta is 4~8 Hz; the filtering bandwidth of Alpha is 8~13 Hz; the filtering bandwidth of Beta is 13~30 Hz; and the filtering bandwidth of Gamma is 30~50 Hz.

[0054] For each frequency band signal, time-domain and frequency-domain features are extracted. Time-domain features include at least one of the following: mean, standard deviation, root mean square, skewness, kurtosis, activity, mobility, and complexity. Frequency-domain features include at least one of the following: absolute power of each physiological rhythm frequency band, five relative powers, absolute power of each frequency band divided by the total power, and spectral centroid. This is used to construct a multimodal EEG feature dataset containing multiple feature views. ,in Indicates the total number of feature views. For the first The feature matrix of a single view; the multimodal feature matrix, where each mode strictly corresponds to one hardware path and one filtering frequency band.

[0055] Multimodal subspace clustering is performed based on a multimodal EEG feature dataset to identify EEG patterns representing different motor intentions, resulting in subspace clustering results distinguished by EEG physical characteristics.

[0056] Generate motion intent decoding instructions for controlling external devices based on the motion intents corresponding to the subspace clustering results;

[0057] Among them, multimodal subspace clustering uses low-rank decomposition to extract global information and introduces a constrained projection matrix to deeply explore the local manifold structure of each view. The subspace clustering results are obtained by solving the following optimization problem:

[0058]

[0059] Where S is the consensus graph set, , It is the feature matrix of the v-th view. consensus graph matrix, This represents the k-th order graph of the v-th view. To select a matrix, Indicates corresponding to Boolean value; Base map, For special diagrams; , , For regularization parameters; It is the identity matrix; Represents graph embedding; For Laplace matrix, It is a degree matrix; It is a matrix whose elements are all 1s.

[0060] Specifically, given the heterogeneous nature of multi-view data, where different views carry unique discriminative information, this invention employs a consensus graph decomposition strategy to decompose the original... It is broken down into two parts: basic diagrams and special diagram sets. Basic diagram Essentially, it acts as a bridge for multi-view data fusion, effectively extracting the feature set shared across views. By introducing nuclear norm constraints during the construction and optimization of the base graph, the principal features are strengthened, clearly presenting crucial information for understanding the overall data structure. Special graphs This approach focuses on revealing the unique characteristics of each view, enabling a more detailed observation of the differences in data distribution across different views and uncovering potential unique information. To accurately preserve this information, the F-norm squared constraint is used as a control mechanism for special plots. The specific representation is as follows:

[0061]

[0062] in, and It is a regularization parameter used to control the complexity of the model.

[0063] Building upon the consensus graph decomposition, higher-order graph techniques are introduced and integrated with the decomposition mechanism. Traditional consensus graphs only model first-order neighborhood relationships of samples, and their sparsity limits their structural expressive power. Higher-order graphs, by introducing multi-order neighborhood contextual information, expand the graph's perceptual range and enhance its representational ability for complex data structures. (Original graph) It can be generated using the KNN algorithm, and then each k-th order graph can be obtained using a higher-order graph construction method. Specifically, it is represented as follows:

[0064] ;

[0065] Based on consensus graph decomposition, higher-order graph fusion, and Laplace rank constraints, this invention achieves its goal by solving the following optimization problem:

[0066] .

[0067] Furthermore, multimodal subspace clustering iteratively updates the variables in the optimization problem using the alternating direction multiplier method. The process continues until the convergence condition is met to obtain the final subspace representation. Based on the final subspace representation, the Laplace rank constraint method is applied to obtain the final clustering result. Specifically, the calculated clustering result y contains 1 or 2, corresponding to whether the robotic arm is gripping or releasing in the 72 tasks.

[0068] Furthermore, the implementation of obtaining the final subspace representation by iteratively updating the optimization problem using the alternating direction multiplier method includes:

[0069] The data for each view is standardized. Initial states for each variable are set, given the number of samples n and cluster 2. The ADMM framework is used to optimize each iterative step:

[0070] S1, Initialize all variables Set the maximum number of iterations. ;

[0071] S2, corresponding to the Lagrange function is as follows:

[0072] ;

[0073] in, , It is the regularization coefficient, and X is a multimodal EEG feature dataset with multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view It is a Lagrange multiplier. To find the trace of the corresponding matrix;

[0074] S3, Update with other variables fixed The subproblem then becomes: And according to The linearly independent row vector group can be further simplified to: In the formula, M= The optimal solution can then be obtained using the KKT conditions.

[0075] S4, Update with other variables fixed : Set matrix J∈ and Let U∈ If it is a block diagonal matrix consisting of identical K×K matrices, then it can be transformed into This subproblem is solved iteratively using a standard quadratic programming solver.

[0076] S5, with other variables fixed, updates B. The subproblem simplifies to: The following optimal solution was obtained using a soft thresholding algorithm: ,in = .

[0077] S6, Update with other variables fixed Its subproblems are: If we set the derivative of this subproblem to zero, then the optimal solution is: .

[0078] S7, fix other variables, update : Optimal yes The eigenvectors corresponding to the first c smallest eigenvalues.

[0079] S8, update while keeping other variables fixed. The update expression is ; ; For the updated parameters ;

[0080] S9, Repeat the parameter update steps until convergence or Stop at this point; obtain the final subspace representation. .

[0081] Furthermore, the present invention also discloses a brainwave data action intention decoding system based on multimodal clustering, which applies the aforementioned brainwave data action intention decoding method based on multimodal clustering. The system includes the following steps:

[0082] The data acquisition module is used to acquire multi-channel EEG data, which includes raw EEG data from multiple leads and electrooculography (EOG) signals from at least one lead.

[0083] The frequency band filtering module, connected to the data acquisition module, is used to perform frequency band filtering on EEG data to obtain multiple frequency band signals corresponding to different EEG physiological rhythms.

[0084] The feature extraction module is used to extract time-domain and frequency-domain features for each frequency band signal to construct a multimodal EEG feature dataset containing multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view;

[0085] The subspace clustering module is used to perform multimodal subspace clustering based on a multimodal EEG feature dataset to identify EEG patterns representing different action intentions and obtain subspace clustering results distinguished by EEG physical characteristics.

[0086] The intent decoding module is used to generate action intent decoding commands for controlling external devices based on the action intents corresponding to the subspace clustering results.

[0087] Furthermore, the data acquisition module is connected to the TGAM brainwave sensor, which acquires bioelectrical information formed by the potential signals of scalp neurons. The data acquisition module converts the bioelectrical information into analog EEG data and then filters out DC drift and ultra-high frequency noise through pre-filtering to obtain the original EEG timing matrix as the original EEG data. The data acquisition module summarizes the original EEG data with the corresponding lead electrooculogram signals to form multi-channel EEG data.

[0088] Experimental Results: The electroencephalogram (EEG) data action intention decoding method based on multimodal clustering of this invention was compared and analyzed with six existing methods. The specific experimental implementation steps are as follows:

[0089] Step 1: Connect the underlying TGAM brainwave sensor to establish a physical coupling channel between the scalp neuron potentials and the acquisition device. The impedance value is used as the physical prior for subsequent MSSC modal weights.

[0090] Step 2: Use the Bluetooth serial port protocol to transmit the acquired operator's bioelectrical signals to the system in real time, and use Bluetooth to transmit amplified simulated EEG signals.

[0091] Step 3: Use software to process the pre-filter: initially filter out DC drift and UHF noise, and output the original EEG timing matrix while retaining the true physical amplitude and phase information;

[0092] Step 4: The host computer receives and aligns the raw EEG data with the timing sequence, and the host computer synchronously receives the raw data from the multi-channel leads;

[0093] Step 5: Software filter group performs layered filtering, using 5 dedicated frequency band filters to filter the original EEG in frequency bands, corresponding to the brain electrophysiological rhythm, and 1 channel for the electrooculogram (EOG) signal;

[0094] Delta: 0.5-4 Hz;

[0095] Theta: 4-8 Hz;

[0096] Alpha: 8-13 Hz;

[0097] Beta: 13-30 Hz;

[0098] Gamma: 30-50 Hz;

[0099] Import the original EEG timing signal and electrooculography (EOG) signal into separate channels;

[0100] Step 6: Use a sliding window to average and remove physical noise;

[0101] Step 7: Multimodal physical feature extraction. Extract multi-view features usable for MSSC from the filtered signals of each frequency band:

[0102] Temporal modalities: mean, standard deviation, root mean square, skewness, kurtosis, activity, mobility, complexity;

[0103] Frequency domain modes: absolute power, 5 relative powers, absolute power of each frequency band divided by total power, 1 spectral centroid);

[0104] Output: Multimodal feature matrix, where each mode strictly corresponds to one hardware path and one filter band.

[0105] Step 9: Perform multimodal subspace clustering (MSSC) and output the clustering results distinguished by EEG physical characteristics (the different results calculated correspond to whether the robotic hand is gripping or releasing in 72 tasks).

[0106] As shown in Table 1, BNCI is a publicly available EEG dataset collected from a patient with a high spinal cord injury. The dataset was developed using a hybrid EEG / EOG brain-computer interface to control a neural prosthesis device. The dataset contains 72 experimental samples, each corresponding to either right-hand motor imagery or a relaxed state. Each sample consists of six different feature views, describing the characteristics of the EEG signals from multiple perspectives, including the time domain, frequency domain, and spatial channels.

[0107] Table 1 shows the dataset used for the experiment.

[0108]

[0109] The required environment for all experiments was: Windows 11 operating system, MATLAB 2021a, Intel Core i9-12900HX CPU, and 16GB RAM. For evaluation metrics, normalized mutual information (NMI) and accuracy (ACC) were used to measure the clustering results in the experiments. The clustering results after 20 experiments were evaluated using normalized mutual information (NMI) as the criterion. Accuracy (ACC) is .

[0110] Compared with other experiments: The six existing methods are as follows: AASC (Affinity aggregation for spectral clustering) is an aggregation spectral clustering method. AWP introduces an adaptive method for weighting views based on clustering ability. MCGC uses a divergence cost function to constrain different graphs. MVGL proposes a graph learning-based method to optimize graph quality, forming a global graph by integrating the optimized graphs through constraining the rank of the Laplacian matrix. SMVSC (Scalable multi-view subspace clustering with unified anchors) unifies anchor learning and graph construction within an optimization framework, enabling the learned anchors to more accurately represent the underlying data distribution. MoMvGC is one of the state-of-the-art methods, achieving the integration of high-order graph selection and weight learning.

[0111] The source code was obtained from the aforementioned prior art research for experimentation, with parameter settings following the authors' recommendations. Each method was run 20 times in a standardized computational environment, and its variability was quantified by averaging the results and standard deviations to arrive at the final results.

[0112] Table 2 Comparison of Accuracy (ACC) Performance

[0113]

[0114] Table 3. Performance Comparison of Normalized Mutual Information (NMI)

[0115]

[0116] Tables 2 and 3 show the comparison results of accuracy (ACC) and normalized mutual information (NMI), with the best results highlighted in bold. Analysis of the results in the tables shows that the method proposed in this invention exhibits the highest or second-highest accuracy among the six compared methods, indicating its ability to correctly assign data points to their respective clusters. This superior performance highlights the effectiveness of decomposing the consensus graph to better integrate basic and specific information. In terms of normalized mutual information, the method proposed in this invention outperforms the other methods; a high NMI score indicates that the obtained clusters are highly consistent with the true labels, reflecting its ability to reveal the true underlying data structure. This robust NMI performance can be attributed to the method's utilization of a higher-order graph structure and the supplementation of specific information for each view, enabling it to more effectively explore the complex structures in multi-view EEG signal data.

[0117] Furthermore, the clustering metric of this invention exhibits low variance, demonstrating the stability and superior performance of the proposed CGDHG method.

[0118] The present invention adopts the above technical solution and has the following beneficial effects compared with the prior art:

[0119] (1) In view of the technical defects of existing methods that tend to ignore the unique information of each view (such as different frequency bands and different feature domains) when fusing multi-view EEG data, resulting in the loss of effective discrimination features, this invention explicitly decomposes the consensus graph into a base graph and a special graph, and introduces a projection matrix specifically used to mine the local manifold structure of each view, which can make more refined and complete use of the complementary information of all views.

[0120] (2) The inherent low signal-to-noise ratio and individual variability of EEG data require the decoding model to have good robustness. Existing methods based on a single consensus graph are not adequately adapted to this. This invention effectively suppresses interference from abnormal noise or artifacts in the signal by constraining the global structural consistency through low-rank decomposition and the sparsity constraint of the projection matrix, and learns a more fundamental EEG pattern representation.

[0121] (3) This invention decomposes the consensus graph into a base graph and special graphs, whereby the base graph is used to learn the consistency structure among the views, while the special graphs are used to capture the local information unique to each view. By introducing corresponding norm constraints, the consensus structure of the base graph is strengthened, and the size of the special graph is controlled to retain unique information.

[0122] This invention achieves a substantial improvement in the accuracy of decoding motor intentions from EEG data, a significant enhancement in system robustness, and a full guarantee of the feasibility of technology implementation, thereby promoting the overall development of high-performance, high-reliability brain-computer interface systems.

[0123] Obviously, the described embodiments are only a part of the embodiments of this application, not all of them. Without conflict, the embodiments and features in the embodiments of this application can be combined with each other. The components of the embodiments of this application described and illustrated herein can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

Claims

1. A method for decoding action intentions from EEG data based on multimodal clustering, characterized in that, Includes the following steps: Acquire multi-channel EEG data, which includes raw EEG data from multiple leads and electrooculography (EOG) signals from at least one lead; Frequency band filtering of EEG data yields multiple frequency band signals corresponding to different EEG physiological rhythms; Temporal and frequency domain features are extracted for each frequency band signal to construct a multimodal EEG feature dataset containing multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view; Multimodal subspace clustering is performed based on a multimodal EEG feature dataset to identify EEG patterns representing different motor intentions, resulting in subspace clustering results distinguished by EEG physical characteristics. Generate motion intent decoding instructions for controlling external devices based on the motion intents corresponding to the subspace clustering results; Among them, multimodal subspace clustering uses consensus graph decomposition technology to extract global information and unique local adjacency information, and introduces high-order graph technology to establish multivariate associations between samples, capturing hidden high-order correlations between views. The subspace clustering results are obtained by solving the following optimization problem: ； Where S is the consensus graph set, , It is the feature matrix of the v-th view. consensus graph matrix, The maximum order of the view; This represents the k-th order graph of the v-th view; To select a matrix, Indicates corresponding to Boolean value; for The base graph based on consensus graph decomposition for A special graph based on consensus graph decomposition; , , For regularization parameters; It is the identity matrix; Represents graph embedding; For Laplace matrix, It is a degree matrix; It is a matrix whose elements are all 1s.

2. The method for decoding action intent from EEG data based on multimodal clustering according to claim 1, characterized in that, The acquisition of multi-channel EEG data includes: The TGAM brainwave sensor acquires the potential signals of neurons in the scalp to form bioelectrical information, which is then transmitted to the processing device and converted into simulated electroencephalogram (EEG) data. The simulated EEG data was pre-filtered to remove DC drift and UHF noise to obtain the original EEG timing matrix as the original EEG data, while retaining the true physical amplitude and phase information; The electrooculogram (EOG) signals from the leads of the raw EEG data are acquired and combined with the raw EEG data to form multichannel EEG data.

3. The method for decoding action intent from EEG data based on multimodal clustering according to claim 1, characterized in that, Frequency band filtering involves filtering the raw EEG data using bandpass filters corresponding to Delta, Theta, Alpha, Beta, and Gamma EEG rhythms, respectively. The filtering bandwidth for Delta is 0.5–4 Hz; for Theta, it is 4–8 Hz; for Alpha, it is 8–13 Hz; for Beta, it is 13–30 Hz; and for Gamma, it is 30–50 Hz.

4. The method for decoding action intent from EEG data based on multimodal clustering according to claim 1, characterized in that, Temporal characteristics include at least one of the following: mean, standard deviation, root mean square, skewness, kurtosis, activity, mobility, and complexity. Frequency domain characteristics include at least one of the absolute power, relative power, and spectral centroid of each physiological rhythm frequency band.

5. The method for decoding action intent from EEG data based on multimodal clustering according to claim 1, characterized in that, Multimodal subspace clustering iteratively updates variables in an optimization problem using the alternating direction multiplier method. This continues until the convergence condition is met to obtain the final subspace representation; The final clustering result is obtained by applying the NCut clustering method based on the final subspace representation.

6. The method for decoding action intent from EEG data based on multimodal clustering according to claim 1, characterized in that, The implementation of obtaining the final subspace representation by iteratively updating the optimization problem using the alternating direction multiplier method includes: S1, Initialize all variables Set the maximum number of iterations. ; S2, corresponding to the Lagrange function is as follows: ； in, , It is the regularization coefficient, and X is a multimodal EEG feature dataset with multiple feature views. ,in, For the first Feature matrix of each view It is a Lagrange multiplier. To find the trace of the corresponding matrix; S3, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ; S4, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ; S5, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ; S6, Update with other variables fixed The corresponding subproblem is: ; S7, update by solving the corresponding subproblems while keeping other variables fixed. The corresponding subproblem is: ; S8, update while keeping other variables fixed. The update expression is ; ; For the updated parameters ; S9, Repeat the parameter update steps until convergence or the number of iterations is reached. Stop at this point; obtain the final subspace representation. .

7. A system for decoding action intent from EEG data based on multimodal clustering, employing the method for decoding action intent from EEG data based on multimodal clustering as described in any one of claims 1 to 6, characterized in that, The system includes the following steps: The data acquisition module is used to acquire multi-channel EEG data, which includes raw EEG data from multiple leads and electrooculography (EOG) signals from at least one lead. The frequency band filtering module, connected to the data acquisition module, is used to perform frequency band filtering on EEG data to obtain multiple frequency band signals corresponding to different EEG physiological rhythms. The feature extraction module is used to extract time-domain and frequency-domain features for each frequency band signal to construct a multimodal EEG feature dataset containing multiple feature views. ,in Indicates the total number of feature views. For the first Feature matrix of each view; The subspace clustering module is used to perform multimodal subspace clustering based on a multimodal EEG feature dataset to identify EEG patterns representing different action intentions and obtain subspace clustering results distinguished by EEG physical characteristics. The intent decoding module is used to generate action intent decoding commands for controlling external devices based on the action intents corresponding to the subspace clustering results.

8. The method for decoding action intent from EEG data based on multimodal clustering according to claim 7, characterized in that, The data acquisition module is connected to the TGAM brainwave sensor, which acquires bioelectrical information formed by the potential signals of scalp neurons. The data acquisition module is used to convert the bioelectrical information into simulated EEG data and then filter out DC drift and ultra-high frequency noise through pre-filtering to obtain the original EEG time series matrix as the original EEG data. The data acquisition module combines the raw EEG data with the corresponding lead electrooculography (EOG) signals to form multichannel EEG data.

Images