Motion imagery recognition method and system of graph convolution network based on tensor decomposition

Abstract

The invention discloses a motor imagery recognition method and system of a graph convolution network based on tensor decomposition. The method comprises the steps of collecting electroencephalogram data when a testee executes a motor imagery task; extracting a time characteristic matrix based on the electroencephalogram data; performing TT tensor decomposition on the electroencephalogram data to obtain a channel factor matrix; determining a Pearson coefficient based on the channel factor matrix to obtain a channel correlation matrix; and inputting the time feature matrix and the channel correlation matrix into a pre-trained graph convolutional network in parallel to obtain a motor imagery recognition result. The time feature matrix and the channel correlation matrix are input into the graph convolutional network in parallel, the graph convolutional network can process the two different types of features at the same time, fully excavate the potential relation between the two different types of features, give play to respective advantages, improve the expression and distinguishing ability of the model to the motor imagery mode, and compared with only using a single feature, the method has the advantages that the efficiency is improved. And the relation between the electroencephalogram signal and the motor imagery task can be more comprehensively described, so that the recognition accuracy is improved.

Description

Technical Field

[0001] This application belongs to the technical field of signal processing for brain-computer interfaces, and specifically relates to a method, system, and storage medium for motion imagination recognition based on a graph convolutional network with tensor decomposition. Background Technique

[0002] Using electroencephalogram (EEG) for motor imagery is one of the important fields in brain-computer interface research. Motor imagery is a paradigm of brain-computer interface, which refers to imagining the movement of specific body parts (such as left and right hands, feet) through the brain without actual action. By collecting the EEG signals generated during the motor imagery process, a brain-computer interface (BCI) system can effectively decode the user's intention. The brain-computer interface technology based on motor imagery has been applied to many fields such as external device control, stroke and epilepsy rehabilitation.

[0003] The methods for motor imagery recognition mainly include traditional machine learning methods and deep learning methods. Traditional machine learning methods cannot fully extract the activities of adjacent brain regions and only focus on the features within electrodes. With the rise of graph convolutional networks, the methods for motor imagery task recognition based on graph convolutional networks effectively solve the limitations of traditional methods. By inputting the original EEG data and the graph structure (correlation coefficients between channels) constructed by electrode modeling into the graph convolutional network, the information transfer between nodes can be effectively realized.

[0004] The currently widely adopted method for constructing the electrode channel graph structure mainly relies on the calculation of Pearson correlation coefficients between channels. Although this strategy reveals the statistical correlation between electrodes to a certain extent, it significantly ignores the intricate and crucial interdependent relationships of EEG signals in the frequency domain and time domain. This simplified processing method inevitably leads to deficiencies in the accuracy and comprehensiveness of the extracted graph structure features, thereby limiting the ability of the graph convolutional network to capture and learn the deep and fine correlation features between electrode channels. Summary of the Invention

[0005] Aiming at the technical problem that the current methods for motor imagery recognition ignore the interdependent relationships of EEG signals in the frequency domain and time domain, thereby limiting the ability of the graph convolutional network to capture and learn the deep and fine correlation features between electrode channels, this application provides a method and system for motor imagery recognition based on a graph convolutional network with tensor decomposition.

[0006] To achieve the above technical objectives, this application adopts the following technical solutions.

[0007] In the first aspect, the embodiments of this application provide a method for motor imagery recognition based on a graph convolutional network with tensor decomposition, including:

[0008] Collect the electroencephalogram (EEG) data of the subject when performing a motor imagery task;

[0009] Extract the time feature matrix based on the EEG data;

[0010] Perform TT tensor decomposition on the EEG data to obtain the channel factor matrix;

[0011] Based on the channel factor matrix, determine the Pearson correlation coefficient to obtain the channel correlation matrix;

[0012] Parallelly input the time feature matrix and the channel correlation matrix into a pre-trained graph convolutional network to obtain the motor imagery recognition result.

[0013] Further, the method further includes preprocessing the EEG data, specifically including:

[0014] Remove artifacts and electrooculogram (EOG) data from the EEG data, and then perform filtering using a fifth-order Butterworth low-pass filter;

[0015] Intercept segment data from the filtered EEG data. Assume the original EEG signal There are a total of N t training sessions, where x i is the EEG data of the i-th training session, x i ∈R C×T , i ∈ (1 to N t ), C is the number of channels, T p is the total number of samples intercepted from the EEG data of N t motor imagery training sessions in time, and R is the number of factors for each channel obtained by decomposition.

[0016] Further, extracting time features based on the EEG data includes:

[0017] Pass the EEG data x i of the i-th training session through the LSTM network in sequence according to the EEG channel order to obtain the time feature matrix The time feature matrix corresponds to the channel order; C is the number of channels, T f is the time feature sequence, and R is the number of factors for each channel obtained by decomposition.

[0018] Further, performing TT tensor decomposition on the EEG data to obtain the channel factor matrix includes:

[0019] Perform wavelet transform on the EEG data x i of the i-th motor imagery training session to convert it into two-dimensional time-frequency data x c ∈R C ×f ; C is the number of channels, and f represents the number of frequency points;

[0020] After scaling and translating the original wavelet basis function, a family of wavelet function bases are obtained;

[0021] The two-dimensional time-frequency data x(t) of the single-channel motion image is transformed into a three-dimensional tensor by performing an inner product operation with the wavelet function basis to realize continuous wavelet transform T p N t The total number of samples captured from the EEG data of each motor imagery training session;

[0022] Using the predetermined TT decomposition objective function, the three-dimensional tensor x is decomposed to obtain the channel factor matrix A∈R C×R , R is the number of factors of each channel obtained by decomposition.

[0023] Further, the graph convolutional network includes a first input layer, a second input layer, a first graph convolutional layer, a second graph convolutional layer, a pooling layer and an output layer;

[0024] The first input layer is used to input the time feature matrix;

[0025] The second input layer, the user inputs the channel correlation matrix, the second input layer and the first input layer are parallel;

[0026] The first graph convolution layer is a first graph network layer for processing the relationship between node features and channel features, the time feature matrix is ​​the node features on the first graph network layer, and the channel correlation matrix is ​​the adjacency matrix on the first graph network layer;

[0027] The first graph network layer is used to process the time feature matrix and the channel correlation matrix to obtain a first graph convolution output;

[0028] The second graph convolution layer is a second graph network layer that processes the relationship between node features and channel features. The time feature matrix passing through the first graph convolution layer is used as the node features of the second graph network layer, and the channel correlation matrix passing through the first graph convolution layer is used as the adjacency matrix on the second graph network layer.

[0029] The second graph convolution layer is used to further mine the deep features between the temporal feature matrix and the channel correlation matrix to obtain a second graph convolution output;

[0030] The pooling layer is used to perform global average pooling on the second graph convolution output features to obtain pooling features;

[0031] The output layer adopts a fully connected layer to map the pooled features to the category space of motion imagery.

[0032] Further, both the first graph convolutional layer and the second graph convolutional layer are two-layer.

[0033] In a second aspect, an embodiment of the present application provides a motor imagery recognition system based on tensor decomposition of a graph convolutional network. The system includes:

[0034] An electroencephalogram (EEG) data acquisition module, configured to acquire EEG data of a subject when performing a motor imagery task;

[0035] A multi-feature extraction module, configured to extract a time feature matrix based on the EEG data; perform TT tensor decomposition on the EEG data to obtain a channel factor matrix; and obtain a channel correlation matrix based on the channel factor matrix by determining Pearson coefficients;

[0036] A motor imagery recognition module, configured to parallelly input the time feature matrix and the channel correlation matrix into a pre-trained graph convolutional network to obtain a motor imagery recognition result.

[0037] Further, the system further includes a data preprocessing module;

[0038] The data preprocessing module is configured to remove artifacts and electrooculogram (EOG) data from the EEG data, and then perform filtering using a fifth-order Butterworth low-pass filter;

[0039] Intercept segment data from the filtered EEG data. Assume the original EEG signal There are a total of N t training times, where x i is the EEG data of the i-th training, x i ∈R C×T , i ∈ (1 to N t ), C is the number of channels, T is the number of EEG data sampling points in the i-th motor imagery training in time, and after preprocessing, N t times of training EEG data where T p is the total number of sampling points intercepted from the EEG data of N t times of motor imagery training in time, and R is the number of factors of each channel obtained by decomposition.

[0040] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the motor imagery recognition method based on tensor decomposition of a graph convolutional network provided by any possible implementation manner of the first aspect are implemented.

[0041] Compared with the prior art, the beneficial technical effects achieved by the motor imagery recognition method and system based on tensor decomposition of graph convolutional network provided by the embodiments of the present application are as follows: By extracting the temporal feature matrix based on EEG data, obtaining the channel factor matrix using TT tensor decomposition, and determining the Pearson coefficient based on this to obtain the channel correlation matrix, this process mines the correlation features between different channels of EEG signals; The temporal feature matrix and the channel correlation matrix are input into the graph convolutional network in parallel. The graph convolutional network can process these two different types of features simultaneously, fully mine the potential connections between them, give full play to their respective advantages, and improve the expression and discrimination ability of the model for motor imagery patterns. Compared with using only a single feature, it can more comprehensively describe the relationship between EEG signals and motor imagery tasks, thereby improving the recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are for illustrative purposes only and are not intended to limit the scope of the disclosure of the present application in any way. Additionally, the shapes and proportional dimensions of the components in the drawings are only schematic for helping the understanding of the present application and do not specifically limit the shapes and proportional dimensions of the components of the present application. Those skilled in the art can, under the teaching of the present application, select various possible shapes and proportional dimensions according to specific circumstances to implement the present application. In the drawings:

[0043] Figure 1 is a schematic flowchart of the motor imagery recognition method based on tensor decomposition of graph convolutional network provided by the embodiments of the present application;

[0044] Figure 2 is a schematic diagram of the network structure implemented by the motor imagery recognition method based on tensor decomposition of graph convolutional network provided by the embodiments of the present application;

[0045] Figure 3 is a schematic diagram of the LSTM structure in the embodiment;

[0046] Figure 4 is a schematic diagram of the TT decomposition effect in the embodiment;

[0047] Figure 5 is a schematic diagram of the structure of the motor imagery recognition system based on tensor decomposition of graph convolutional network provided by the embodiments of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0048] In order to enable those skilled in the art to better understand the technical solutions in the present application, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0049] In the description of the present application, the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more features.

[0050] In order to comprehensively cover the brain regions related to electroencephalogram (EEG) activities, generally, when collecting EEG signals, experimenters will use a relatively large number of electrodes to collect EEG signals. However, not all lead data can provide effective decoding and classification information. For example, in a visual evoked BCI experiment, the lead channels far from the visual cortex cannot be well used to identify visual information, and there will also be redundant and noisy data introduced. Channel selection before the classification task can reduce irrelevant noise and redundant information, thereby reducing the computational complexity and improving the performance of the BCI system. In order to effectively remove the useless and redundant channels in the motor imagery task, a commonly used method is to use the Pearson correlation coefficient to select the channels related to the MI task. However, this channel selection method often cannot fully combine the spatial and temporal information between channels.

[0051] Combined with Figure 1 , the embodiment of the present application provides a method for motor imagery recognition based on tensor decomposition of graph convolutional network, and the method includes the following steps:

[0052] Step 1: Collect EEG data when the subject performs a motor imagery task;

[0053] Step 2: Extract a time feature matrix based on the EEG data;

[0054] Step 3: Perform TT tensor decomposition on the EEG data to obtain a channel factor matrix; determine the Pearson coefficient based on the channel factor matrix to obtain a channel correlation matrix;

[0055] Step 4: Parallelly input the time feature matrix and the channel correlation matrix into a pre-trained graph convolutional network to obtain a motor imagery recognition result.

[0056] In the specific implementation process, an EEG acquisition device can be used to collect the EEG data of the subject. Commonly used ones are EEG acquisition devices based on wet electrodes, dry electrodes or semi-dry electrodes.

[0057] In some embodiments, the method further includes preprocessing the EEG data, specifically including:

[0058] Remove artifacts from EEG data and EOG data. The frequency of the EEG signals stimulated by motor imagery mainly concentrates in the lower frequency band of 7-35 Hz. Then, a fifth-order Butterworth low-pass filter is selected for filtering to remove higher frequencies, so as to reduce misclassification caused by noise interference, thereby significantly improving the recognition accuracy of motor imagery.

[0059] Intercept segment data from the filtered EEG data. Assume the original EEG signal There are N t times of motor imagery training. Among them, x i is the EEG data of the i-th motor imagery training, and x i ∈R C×T , i ∈ (1~N t ), C is the number of channels, T is the number of sampling points of the EEG data in the i-th motor imagery training in time. The sampling frequency can be 250 Hz. After preprocessing, N t times of motor imagery training EEG data where T p is the total number of samples intercepted from the EEG data of N t times of motor imagery training in time, and R is the number of factors decomposed for each channel.

[0060] In the embodiment, step 2 can perform temporal feature extraction by inputting the original EEG data into a selectable LSTM or residual network to obtain temporal features.

[0061] In some embodiments, extracting a temporal feature matrix based on EEG data includes: sequentially passing the EEG data x i of the i-th motor imagery training through an LSTM network according to the EEG channel order to obtain a temporal feature matrix The temporal feature matrix corresponds to the channel order. C is the number of channels, T f is the temporal feature sequence, and R is the number of factors decomposed for each channel.

[0062] Please refer to Figure 2 , the temporal feature matrix and the channel correlation matrix will enter the network structure in parallel. The upper branch is to perform temporal feature extraction on the motor imagery data. The selected feature extraction network is a long short-term memory network. The specific structure can be referred to Figure 3 , compared with using the original data, the temporal data first passes through the LSTM network, and more effective temporal features can be obtained, which is helpful for the classification of the motor imagery (MI) task. The LSTM network effectively captures and retains the long-term dependence information in the time series data by introducing a gating mechanism, overcoming the limitations of the standard recurrent neural network in dealing with long-term dependence problems. The process of the LSTM learning the temporal feature sequences before and after to obtain the temporal feature matrix is as follows:

[0063] Step 201: Input x t and the output hidden state h at the previous moment t-1 are input into the input gate to calculate a new information i to be retained t and a new state information The specific calculation formula is as follows:

[0064] i t = σ(Wi[h t-1 , x t + b i );

[0065]

[0066] Step 202: Input x t and the output h at the previous moment t-1 are input into the forget gate to calculate an old information f to be forgotten t , and the specific calculation formula is as follows:

[0067] f t = σ(W f [h t-1 , x t + b f );

[0068] Step 203: The new state information c t can be calculated through the state c at the previous moment t-1 and the output of the input gate as follows:

[0069]

[0070] Step 204: Input x t and the output g at the previous moment t-1 are passed into the output gate to calculate a new feature o to be output t :

[0071] o t = σ(W o [h t-1 , x t + b o );

[0072] Step 205: The output of the LSTM is to pass the new state c t and the output o of the output gate t into the activation function, and the calculation formula is as follows:

[0073] h t = o t tanh(c t );

[0074] In the above formula, x represents the input vector, y represents the output vector, t represents time, W represents the weight matrix, b represents the bias matrix, σ represents the activation function of sigmoid, tanh represents the activation function of tanh, and i, f, and o represent the input gate, forget gate, and output gate respectively.

[0075] The LSTM network plays a very good role in processing sequence data and can effectively retain and learn information over a longer time span. Motor imagery data Sequentially passes through the LSTM network according to the order of EEG channels to obtain a time feature matrix Corresponding to the channel order.

[0076] In the embodiment, step 3 includes constructing a three-dimensional tensor, building a tensor decomposition model, and solving the channel factor matrix.

[0077] In the embodiment, the frequency information of the EEG signal can be obtained through Fourier transform or wavelet transform. Compared with Fourier transform which uses a set of infinitely long trigonometric function bases for signal fitting, wavelet transform uses a set of orthogonal and rapidly decaying wavelet functions for signal fitting and can obtain the positions of signal components with different frequencies in the time domain.

[0078] The motor imagery EEG data for a single training is where C is the number of channels, T p is the total number of samples intercepted from the EEG data of N t times of motor imagery training in time. Perform wavelet transform on the EEG data for a single training and convert its data into two-dimensional time-frequency data of x c ∈R C×f .

[0079] In some embodiments, the selected wavelet basis function ψ(t) is the Daubechies wavelet. Compared with other basis functions, it is more suitable for multi-dimensional data such as EEG signals and can analyze better time and space localization features. The frequency range of wavelet transform is set between 7 Hz and 35 Hz, covering the main EEG frequencies.

[0080] Performing scale transformation and time shift on the wavelet function will generate a family of wavelet function bases:

[0081]

[0082] ψ(t) is the wavelet basis function, a is the scale parameter, and b is the translation parameter.

[0083] Given a single-channel motor imagery EEG signal x(t), its continuous wavelet transform is the inner product of x(t) and ψ a，b (t):

[0084]

[0085] Through wavelet transform, the motor imagery data of a single experiment is converted into a three-dimensional tensor where f represents the number of frequency points.

[0086] As Figure 4 shown, performing TT decomposition on the three-dimensional tensor will yield three factor matrices, namely the channel factor matrix A ∈ R C×R , the frequency factor matrix B ∈ R R×f×R and the time factor matrix

[0087] x c，f，T ≈a 1 b 1 c 1 +a 2 b 2 c 2 +a 3 b 3 c 3 = A · B · C;

[0088] To optimize these three factor matrices, we construct the following objective function for TT decomposition:

[0089]

[0090] Regularization terms λ 1 , λ 2 , λ 3 are added to the objective function to avoid overfitting and ensure that the decomposed matrices have smaller norms. We continuously optimize the three factor matrices by the Stochastic Gradient Descent (SGD) method to minimize the objective function of tensor decomposition.

[0091] Through fine TT tensor decomposition of EEG data, the embodiment can extract the channel factor matrix containing deep spatial domain features. This channel factor matrix not only captures the unique characteristics of a single channel but also incorporates the interaction information between channels; comprehensive feature capture helps to more accurately depict the neural activity state of the brain during motor imagery, thereby improving the recognition accuracy.

[0092] The channel factor matrix A ∈ R C×R integrates spatial, frequency, and time domain information, and uses Pearson correlation analysis to rank the importance of each channel. The Pearson correlation coefficient can quantify the correlation between two or more random variables and can be defined as:

[0093]

[0094] Where X and Y are two observable variables, n is the number of observations, and are the means of the two variables, and σ X and σ Y are the standard deviations of the two variables. The value range of the correlation coefficient is between 0 and 1. After calculating the Pearson correlation coefficient for the channel factor matrix A, the channel correlation matrix P ∈ R C×C for a single trial can be obtained. For N t training data, a total channel correlation matrix Q ∈ R C×C×Nt is obtained, and this channel correlation matrix will be fed into the graph convolutional network as the graph structure of the motor imagery signal for learning and classification.

[0095] As an example, each EEG channel can be regarded as a node of the graph. If the EEG data contains channels, there will be nodes in the graph. These nodes represent the EEG signal acquisition points corresponding to different brain regions, and each node carries the relevant information of the EEG signal of the corresponding channel. The element values in the channel correlation matrix directly determine the weights of the edges in the graph. For channel and channel, their coefficients in the channel correlation matrix are the weights of the edge connecting node and node. The larger the weight, the stronger the correlation between these two channels, and the closer their connection in the graph structure. In this way, the channel correlation matrix completely defines the topological structure of the graph and the weight distribution of the edges.

[0096] In the embodiment, the original data is dimensionally elevated through wavelet packet decomposition, including channel, time series, and frequency information, and then the channel factor matrix is obtained through TT tensor decomposition. Calculate the correlation coefficient between channels for the channel factor matrix to obtain the graph structure, so as to realize the time-varying EEG signals on different channels and have different frequency components. Through the fusion of multi-dimensional information, the characteristics of EEG signals can be comprehensively captured, which helps to reveal the potential dependence relationship between different EEG channels. The graph structure can help identify the channels that play a key role in the motor imagery task.

[0097] In a specific embodiment, in step 4, the time feature matrix and the channel correlation matrix are input into the graph convolutional network for node classification.

[0098] As an example, the graph convolutional network includes a first input layer, a second input layer, a first graph convolutional layer, a second graph convolutional layer, a pooling layer, and an output layer; the first input layer is used to input a time feature matrix; the second input layer is for the user to input a channel correlation matrix; the second input layer is parallel to the first input layer; the first graph convolutional layer is the first graph network layer for processing the relationship between node features and channel features, the time feature matrix is the node feature on the first graph network layer, and the channel correlation matrix is the adjacency matrix on the first graph network layer; the first graph network layer is used to process the time feature matrix and the channel correlation matrix to obtain a first graph convolutional output; the second graph convolutional layer is the second graph network layer for processing the relationship between node features and channel features, the time feature matrix after passing through the first graph convolutional layer is used as the node feature of the second graph network layer, and the channel correlation matrix after passing through the first graph convolutional layer is used as the adjacency matrix on the second graph network layer; the second graph convolutional layer is used to further extract the deep features between the time feature matrix and the channel correlation matrix to obtain a second graph convolutional output; the pooling layer is used to perform global average pooling on the second graph convolutional output features to obtain pooling features; the output layer uses a fully connected layer to map the pooling features to the category space of motor imagery.

[0099] In the embodiment, a graph convolutional network is used to combine time and space features, and a parallel network structure is adopted, which can combine time-domain and space-domain information. Compared with traditional methods, it can effectively correlate the activities of different brain regions.

[0100] Based on the same inventive concept as the motor imagery recognition method based on tensor decomposition of the graph convolutional network provided in the above embodiments, the embodiments of the present application also provide a motor imagery recognition system based on tensor decomposition of the graph convolutional network, as Figure 5 shown, including an electroencephalogram data acquisition module, a multi-feature extraction module, and a motor imagery recognition module. Among them. The electroencephalogram data acquisition module is used to collect electroencephalogram data when the subject performs a motor imagery task; the multi-feature extraction module is used to extract a time feature matrix based on the electroencephalogram data; perform TT tensor decomposition on the electroencephalogram data to obtain a channel factor matrix; based on the channel factor matrix, determine the Pearson coefficient to obtain a channel correlation matrix; the motor imagery recognition module is used to parallelly input the time feature matrix and the channel correlation matrix into a pre-trained graph convolutional network to obtain a motor imagery recognition result.

[0101] Furthermore, as Figure 5 shown, the system further includes a data preprocessing module; the data preprocessing module is used to remove artifacts and electrooculogram data from the electroencephalogram data, and then perform filtering using a fifth-order Butterworth low-pass filter;

[0102] Intercept segment data from the filtered electroencephalogram data, assuming the original electroencephalogram signal There are a total of N t times of training, where xi is the EEG data for the i-th training, x i ∈R C×T , i ∈ (1 to N t ), C is the number of channels, T is the number of EEG data sampling points for the i-th motor imagery training in time. After preprocessing, N t times of training EEG data where T p is the total number of samples intercepted from the EEG data of N t times of motor imagery training in time, and R is the number of factors for each channel obtained by decomposition.

[0103] For the specific limitations of the motor imagery recognition system based on tensor decomposition graph convolutional network, reference can be made to the limitations of the motor imagery recognition method based on tensor decomposition graph convolutional network in the above text, which will not be elaborated here. Each module in the above motor imagery recognition system based on tensor decomposition graph convolutional network can be implemented in whole or in part by software, hardware and their combination. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so as to facilitate the processor to call and execute the operations corresponding to the above modules.

[0104] The embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the motor imagery recognition method based on tensor decomposition graph convolutional network provided in the above embodiments are implemented.

[0105] The above has introduced in detail the motor imagery recognition method, system and medium based on tensor decomposition graph convolutional network provided by the present application. Specific examples are used in this article to elaborate the principle and implementation manner of the present application. The description of the above embodiments is only used to help understand the concept of the present application, and should not be construed as a limitation on the protection scope of the present application.

Claims

1. A motion imagery recognition method based on a graph convolutional network with tensor decomposition, characterized in that: include: Collect EEG data of subjects while they perform motor imagery tasks; Extracting a temporal feature matrix based on the EEG data; Performing TT tensor decomposition on the EEG data to obtain a channel factor matrix; Obtaining a channel correlation matrix by determining the Pearson coefficient based on the channel factor matrix; The temporal feature matrix and the channel correlation matrix are input in parallel into a pre-trained graph convolutional network to obtain a motor imagery recognition result.

2. The method for motion imagery recognition based on graph convolutional network based on tensor decomposition according to claim 1, characterized in that: The method further includes preprocessing the EEG data, specifically including: Remove artifacts from the EEG data and remove electrooculogram data, and then filter the data using a fifth-order Butterworth low-pass filter; The filtered EEG data is intercepted into fragment data, assuming that the original EEG signal Total N t Motor imagery training, where x i is the EEG data of the ith motor imagery training, x i ∈R C×T , i∈(1~N t ), C is the number of channels, T is the number of EEG data sampling points of the i-th motor imagery training in time, and N is obtained after preprocessing. t Training EEG data T p N t The total number of samples intercepted from the EEG data of the motor imagery training, R is the number of factors of each channel obtained by decomposition.

3. The method for motion imagery recognition based on graph convolutional network based on tensor decomposition according to claim 1, characterized in that: Extracting a time feature matrix based on the EEG data includes: The EEG data x of the i-th motor imagery training i According to the order of EEG channels, the time feature matrix is ​​obtained by passing through the LSTM network in sequence. The time feature matrix corresponds to the channel order; C is the number of channels, T f is the time feature sequence, and R is the number of factors of each channel obtained by decomposition.

4. The method for motion imagery recognition based on graph convolutional network based on tensor decomposition according to claim 1, characterized in that: Performing TT tensor decomposition on the EEG data to obtain a channel factor matrix includes: For the EEG data x of the i-th motor imagery training i Perform wavelet transformation and convert it into two-dimensional time-frequency data x c ∈R C×f ; C is the number of channels, f is the number of frequency points; After scaling and translating the original wavelet basis function, a family of wavelet function bases are obtained; The two-dimensional time-frequency data x(t) of the single-channel motion image is transformed into a three-dimensional tensor by performing an inner product operation with the wavelet function basis to realize continuous wavelet transform T p N t The total number of samples captured from the EEG data of each motor imagery training session; Using the predetermined TT decomposition objective function, the three-dimensional tensor x is decomposed to obtain the channel factor matrix A, A∈R C×R , R is the number of factors of each channel obtained by decomposition.

5. The method for motion imagery recognition based on graph convolutional network based on tensor decomposition according to claim 1, characterized in that: The graph convolutional network includes a first input layer, a second input layer, a first graph convolutional layer, a second graph convolutional layer, a pooling layer and an output layer; The first input layer is used to input the time feature matrix; The second input layer, the user inputs the channel correlation matrix, the second input layer and the first input layer are parallel; The first graph convolution layer is a first graph network layer for processing the relationship between node features and channel features, the time feature matrix is ​​the node features on the first graph network layer, and the channel correlation matrix is ​​the adjacency matrix on the first graph network layer; The first graph network layer is used to process the time feature matrix and the channel correlation matrix to obtain a first graph convolution output; The second graph convolution layer is a second graph network layer that processes the relationship between node features and channel features. The time feature matrix passing through the first graph convolution layer is used as the node features of the second graph network layer, and the channel correlation matrix passing through the first graph convolution layer is used as the adjacency matrix on the second graph network layer. The second graph convolution layer is used to further mine the deep features between the temporal feature matrix and the channel correlation matrix to obtain a second graph convolution output; The pooling layer is used to perform global average pooling on the second graph convolution output features to obtain pooling features; The output layer adopts a fully connected layer to map the pooled features to the category space of motion imagery.

6. The method for motion imagery recognition based on tensor decomposition graph convolutional network according to claim 5, characterized in that: The first graph convolution layer and the second graph convolution layer both have two layers.

7. A motion imagery recognition system based on a tensor decomposition graph convolutional network, characterized in that: The system comprises: The EEG data acquisition module is used to collect EEG data of the subjects when they perform motor imagery tasks; A multi-feature extraction module is used to extract a time feature matrix based on the EEG data; perform TT tensor decomposition on the EEG data to obtain a channel factor matrix; and obtain a channel correlation matrix based on the channel factor matrix by determining the Pearson coefficient; The motor imagery recognition module is used to input the time feature matrix and the channel correlation matrix in parallel into a pre-trained graph convolutional network to obtain a motor imagery recognition result.

8. The motor imagery recognition system based on tensor decomposition and graph convolutional network according to claim 7, characterized in that: The system also includes a data preprocessing module; The data preprocessing module is used to remove artifacts and electrooculogram data from the EEG data, and then filter the data using a fifth-order Butterworth low-pass filter; The filtered EEG data is intercepted into fragment data, assuming that the original EEG signal Total N t training times, where x i is the EEG data of the i-th training, x i ∈R C×T , i∈(1~N t ), C is the number of channels, T is the number of EEG data sampling points of the i-th motor imagery training in time, and N is obtained after preprocessing. t Training EEG data Where T p N t The total number of samples intercepted from the EEG data of the motor imagery training, R is the number of factors of each channel obtained by decomposition.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the motion imagery recognition method based on a graph convolutional network based on tensor decomposition as described in any one of claims 1 to 6 are implemented.

Images