Motor imagery electroencephalogram signal decoding method and device based on brain region connection relationship, terminal, medium and product

Abstract

The application provides a motor imagery electroencephalogram signal decoding method and device based on brain region connection relationship, a terminal, a medium and a product. A plurality of collected multi-category motor imagery task electroencephalogram signals are respectively preprocessed to obtain an electroencephalogram signal dataset; a motor imagery electroencephalogram signal decoding model based on brain region connection relationship is constructed; the obtained electroencephalogram signal dataset is used to train, verify and test the motor imagery electroencephalogram signal decoding model, so as to output a corresponding decoding result after inputting a motor imagery task electroencephalogram signal to be decoded. The node relationship and dynamic topological relationship between electroencephalogram electrodes are fully considered, the information amount of the extracted features of the model is enriched, and the landing application of the motor imagery brain-computer interface technology is promoted.

Description

Technical Field

[0001] This application relates to the field of brain signal processing technology, and in particular to a method, device, terminal, medium and product for decoding motor imagery EEG signals based on brain region connectivity. Background Technology

[0002] Brain-computer interfaces (BCIs) are human-computer interaction technologies designed to directly connect a user's brain with the external world. Electroencephalography (EEG), a non-invasive technique that records brain electrophysiological activity by measuring electrical signals on the scalp, offers advantages such as high temporal resolution and high user acceptance. Motor imagery (MI) is an EEG-based BCI paradigm. In motor imagery tasks, users receive cues about a specific movement and are required to continuously imagine themselves performing that movement for a given period. Although the user is not actually moving, specific brain regions still exhibit event-related desynchronization (ERD) and event-related synchronization (ERS) phenomena similar to actual movement. Through these phenomena, motor imagery-based BCI systems can generate various control commands to control external devices, such as brain-controlled wheelchairs and robotic arms.

[0003] Typically, brain-computer interface systems based on motor imagery use dozens of electrodes to collect motor imagery signals from multiple brain regions. Accurately processing the dynamic topological relationships between electrodes is crucial for understanding brain function. However, most existing deep learning-based motor imagery decoding methods (such as EEG networks) only use traditional convolutional operations to extract features, essentially weighting and summing the signals from each electrode, while neglecting the dynamic connectivity between brain regions. From a graph theory perspective, this approach only focuses on the individual nodes, ignoring the connections between them, resulting in low information content in the extracted features and low accuracy in decoding motor imagery tasks. Summary of the Invention

[0004] In view of the shortcomings of the prior art described above, the purpose of this application is to provide a method, device, terminal, medium and product for decoding motor imagery EEG signals based on brain region connectivity, so as to solve the problems in the prior art.

[0005] To achieve the above and other related objectives, the first aspect of this application provides a method for decoding motor imagery EEG signals based on brain region connectivity, comprising: preprocessing multiple multi-category motor imagery task EEG signals to obtain an EEG signal dataset; constructing a motor imagery EEG signal decoding model based on brain region connectivity; and using the obtained EEG signal dataset to train, verify, and test the motor imagery EEG signal decoding model so as to output the corresponding decoding result after inputting the motor imagery task EEG signal to be decoded.

[0006] In some embodiments of the first aspect of this application, the specific preprocessing process includes: sequentially performing filtering, downsampling, and baseline removal processing on each acquired multi-category motor imagery task EEG signal to obtain each pre-processed EEG signal; segmenting each pre-processed EEG signal according to the corresponding motor imagery task time period to obtain multiple motor imagery task EEG signals; and labeling each motor imagery task EEG signal to obtain an EEG signal dataset.

[0007] In some embodiments of the first aspect of this application, the constructed motor imagery EEG signal decoding model based on brain region connectivity includes: a temporal module, a connectivity encoding module, a spatial module, a depth-separable module, and a prediction module connected in sequence; wherein, the temporal module includes: a first batch of normalized layers connected to a temporal convolutional layer; the spatial module includes: a spatial convolutional layer, a second batch of normalized layers, a first activation layer, a first average pooling layer, and a first random deactivation layer connected in sequence; the depth-separable module includes: a depth-separable convolutional layer, a third batch of normalized layers, a second activation layer, a second average pooling layer, and a second random deactivation layer connected in sequence; the temporal module, the connectivity encoding module, the spatial module, the depth-separable module, and the prediction module sequentially process the motor imagery task EEG signals in the EEG signal dataset to output corresponding decoding results.

[0008] In some embodiments of the first aspect of this application, the connectivity encoding module is used to encode the features extracted by the time-domain module from the EEG signal of the motor imagery task to obtain brain region connectivity features; wherein, the specific process of connectivity encoding includes: calculating the neighbor relationship matrix, spatial relationship matrix, routing relationship matrix, and central relationship matrix between electrodes based on matrix calculation rules according to the electrodes used in the process of collecting the EEG signal of the motor imagery task; adding the neighbor relationship matrix, spatial relationship matrix, and routing relationship matrix and normalizing them to obtain a normalized matrix; multiplying the features extracted by the time-domain module with the normalized matrix, and performing a Hadamard product with the central relationship matrix to obtain brain region connectivity features.

[0009] In some embodiments of the first aspect of this application, the matrix calculation rules include: a neighbor relationship matrix calculation rule, comprising: initializing a first all-zero connection matrix; wherein the number of rows and columns of the first all-zero connection matrix are equal to the number of electrodes; for each electrode, setting the values ​​of the corresponding positions of each of the electrode's neighboring electrodes in the first all-zero connection matrix to 1; outputting the matrix obtained after setting the values ​​of all electrodes as a neighbor relationship matrix; a spatial relationship matrix calculation rule, comprising: initializing a second all-zero connection matrix; wherein the number of rows and columns of the second all-zero connection matrix are equal to the number of electrodes; calculating the distance between each electrode according to the coordinates of each electrode, and filling the calculated distance between each electrode into the second all-zero connection matrix to obtain a distance matrix; quantizing the distance matrix, and performing spatial embedding encoding on the quantized distance matrix to obtain a spatial relationship matrix; a routing relationship matrix calculation rule, comprising: initializing a third all-zero connection matrix; wherein, The number of rows and columns in the third all-zero connection matrix are equal to the number of electrodes. Based on the distance matrix, the shortest path from each electrode to the other electrodes is calculated using the Floyd algorithm. The shortest path of each electrode is encoded according to a set encoding structure to obtain the path features of each electrode. Routing embedding encoding is performed on the path features of all electrodes, and the result of the routing embedding encoding is written into the third all-zero connection matrix to obtain the routing relation matrix. The set encoding structure includes: whether brain region transfer is performed, the smaller brain region number among the two brain region numbers of the two electrodes forming the shortest path, and the larger brain region number among the two brain region numbers of the two electrodes forming the shortest path. The central relation matrix calculation rules include: initializing a column matrix; where the number of rows in the column matrix is ​​equal to the number of electrodes, and the number of columns is equal to 1; calculating the degree of adjacent nodes for each electrode; performing central embedding encoding on the degree of adjacent nodes for all electrodes, and writing the result of the central embedding encoding after normalization into the column matrix to obtain the central relation matrix.

[0010] In some embodiments of the first aspect of this application, a motor imagery EEG signal decoding model is trained and validated using the obtained EEG signal dataset. The process includes: inputting the training set from the EEG signal dataset into the motor imagery EEG signal decoding model for multiple rounds of training; comparing the decoding results obtained after each round of training with the labels in the training set, calculating the loss function, and backpropagating the loss function to update the model parameters of the motor imagery EEG signal decoding model; evaluating the decoding model after each round of model parameter updates using the validation set from the EEG signal dataset, and using the decoding model with the highest validation accuracy as the final motor imagery EEG signal decoding model; and evaluating the final motor imagery EEG signal decoding model using the test set from the EEG signal dataset.

[0011] To achieve the above and other related objectives, a second aspect of this application provides a motor imagery EEG signal decoding device based on brain region connectivity, comprising: an acquisition and processing module for preprocessing multiple acquired multi-category motor imagery task EEG signals to obtain an EEG signal dataset; a model building module connected to the acquisition and processing module for constructing a motor imagery EEG signal decoding model based on brain region connectivity; and a model training module connected to the model building module for training, validating, and testing the motor imagery EEG signal decoding model using the obtained EEG signal dataset, so as to output the corresponding decoding result after inputting the motor imagery task EEG signal to be decoded.

[0012] To achieve the above and other related objectives, a third aspect of this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method for decoding motor imagery EEG signals based on brain region connectivity.

[0013] To achieve the above and other related objectives, a fourth aspect of this application provides a computer program product comprising computer program code that, when executed on a computer, enables the computer to implement the method for decoding motor imagery EEG signals based on brain region connectivity.

[0014] To achieve the above and other related objectives, a fifth aspect of this application provides an electronic terminal, including a memory, a processor, and a computer program stored in the memory; the processor executes the computer program to implement the method for decoding motor imagery EEG signals based on brain region connectivity.

[0015] As described above, the method, device, terminal, medium, and product for decoding motor imagery EEG signals based on brain region connectivity in this application have the following beneficial effects:

[0016] This application fully considers the node relationships and dynamic topological relationships between brain electrodes, enriches the information content of the features extracted by the model, and promotes the practical application of motor imagery brain-computer interface technology. Attached Figure Description

[0017] Figure 1 The diagram shown is a flowchart illustrating a method for decoding motor imagery EEG signals based on brain region connectivity in one embodiment of this application.

[0018] Figure 2 The diagram shown is a schematic representation of a motor imagery EEG signal decoding model based on brain region connectivity in one embodiment of this application.

[0019] Figure 3 The diagram shown is a schematic representation of the neighbor relationship matrix between electrodes in one embodiment of this application.

[0020] Figure 4 The diagram shown is a flowchart illustrating the process of obtaining brain region connectivity features in one embodiment of this application.

[0021] Figure 5 The diagram shows the spatial distribution of 22-lead electrodes and brain region numbering in one embodiment of this application.

[0022] Figure 6 The figure shown is a comparison of the classification accuracy of a motor imagery signal decoding method based on brain region connectivity in one embodiment of this application with the accuracy of two other decoding methods.

[0023] Figure 7 The diagram shown is a schematic representation of a motor imagery EEG signal decoding device based on brain region connectivity in one embodiment of this application.

[0024] Figure 8 The diagram shown is a structural schematic of an electronic terminal according to an embodiment of this application. Detailed Implementation

[0025] The following specific examples illustrate the implementation of this application. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be noted that, unless otherwise specified, the following embodiments and features in the embodiments can be combined with each other.

[0026] In the embodiments of this application, terms such as "first" and "second" are used to distinguish identical or similar items with essentially the same function and purpose. For example, the first all-zero connection matrix and the second all-zero connection matrix are merely used to distinguish different all-zero connection matrices and do not limit their order. Those skilled in the art will understand that terms such as "first" and "second" do not limit the quantity or execution order, and that terms such as "first" and "second" do not necessarily imply that they are different.

[0027] It should be noted that, in the embodiments of this application, the words "exemplary" or "for example" indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in this application should not be construed as being more preferred or advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0028] In this application embodiment, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0029] Before providing a further detailed description of the present invention, the nouns and terms used in the embodiments of the present invention are explained, and the nouns and terms used in the embodiments of the present invention are subject to the following interpretations:

[0030] <1> Motor imagery is a mental training technique that involves visualizing yourself performing a movement or action in your mind without actually doing so. This technique has applications in multiple fields, including sports training, rehabilitation, neuroscience, and psychology.

[0031] <2> Floyd's algorithm, also known as the Floyd-Warshall algorithm, is an algorithm used to find the shortest path between all pairs of vertices in a weighted graph. It is suitable for dense graphs, that is, graphs where the number of edges is close to the square of the number of vertices.

[0032] <3> Embedding encoding typically refers to the process of converting data objects (such as words, sentences, images, or video frames) into numerical vectors. These vectors can capture certain features or attributes of the original data and be used in machine learning models to improve the model's ability to understand the data.

[0033] <4> Depthwise separable convolution is an optimization technique used in convolutional neural networks (CNNs), especially when computational resources are limited. This technique can significantly reduce the computational cost and number of parameters of the model while maintaining similar or improved performance.

[0034] <5> Weight mapping typically refers to the process in a neural network of how input data is transformed into output through weights (i.e., model parameters).

[0035] <6> The Hadamard product, also known as the element-wise product or Schurproduct, is an operation between two arrays.

[0036] This application provides a method, device, terminal, medium, and product for decoding motor imagery EEG signals based on brain region connectivity. It preprocesses multiple collected EEG signals from various motor imagery tasks to obtain EEG signal datasets; constructs a motor imagery EEG signal decoding model based on brain region connectivity; and trains, validates, and tests the decoding model using the obtained EEG signal dataset to output corresponding decoding results after inputting the motor imagery task EEG signal to be decoded. This application fully considers the node relationships and dynamic topological relationships between EEG electrodes, enriching the information content of the features extracted by the model and promoting the practical application of motor imagery brain-computer interface technology.

[0037] To facilitate understanding of the embodiments of this application, firstly, in conjunction with Figure 1 Detailed explanation. Figure 1 This document illustrates a flowchart of a method for decoding motor imagery EEG signals based on brain region connectivity, according to an embodiment of the present invention. The method for decoding motor imagery EEG signals based on brain region connectivity in this embodiment mainly includes the following steps:

[0038] Step S101: Preprocess the collected EEG signals from multiple multi-category motor imagery tasks to obtain an EEG signal dataset.

[0039] In one embodiment, the EEG signal dataset is divided into a training set, a validation set, and a test set.

[0040] In one specific embodiment, the EEG signals captured by the acquisition device for each subject during a multi-class motor imagery task constitute a multi-class motor imagery task EEG signal.

[0041] In one specific embodiment, the types of motor imagery tasks include left hand, right hand, both feet, and tongue, respectively.

[0042] In one embodiment, the specific preprocessing process includes: sequentially performing filtering, downsampling, and baseline removal processing on each acquired multi-category motor imagery task EEG signal to obtain each pre-processed EEG signal; segmenting each pre-processed EEG signal according to the corresponding motor imagery task time period to obtain multiple motor imagery task EEG signals; and labeling each motor imagery task EEG signal to obtain an EEG signal dataset.

[0043] It should be noted that the purpose of downsampling is to reduce the amount of data.

[0044] In one specific embodiment, the preprocessing process includes:

[0045] The first step is to perform a 30-40Hz bandpass filter on the collected EEG signals for each multi-category motor imagery task.

[0046] The second step is to reduce the sampling rate of the filtered signal obtained in the first step from 1000Hz to 250Hz.

[0047] The third step is to calculate the mean of the EEG signal of the corresponding subject 200ms before performing the motor imagery task, and use this mean as the baseline signal; the baseline signal includes the baseline channel signal with the same number of EEG signal channels as the multi-class motor imagery task; subtract the corresponding baseline channel signal from each channel signal of the signal with reduced sampling rate in the second step to obtain the signal with the baseline subtracted;

[0048] The fourth step is to segment the signals obtained in the third step (after subtracting the baseline) according to the corresponding time periods of the motor imagery task to obtain multiple EEG signals for the motor imagery task.

[0049] The fifth step is to label the EEG signals for each motor imagery task to obtain an EEG signal dataset.

[0050] It should be noted that since the EEG signals for each multi-category motor imagery task are obtained through continuous acquisition, each multi-category motor imagery task EEG signal includes EEG signals from multiple categories of motor imagery tasks as well as EEG signals when not performing motor imagery tasks. Therefore, it is necessary to segment the multi-category motor imagery task EEG signals to obtain the EEG signals when performing motor imagery tasks.

[0051] To better illustrate how the EEG signal dataset is acquired, a specific example is provided below.

[0052] Example 1: A method for acquiring electroencephalogram (EEG) signal datasets.

[0053] Nine subjects were selected, and each subject performed two batches of four-category motor imagery tasks, 288 times per batch. EEG signals from each subject were captured using a 22-lead EEG acquisition device with a 1000Hz sampling rate during the four-category motor imagery tasks. The EEG signals from the first batch of four-category motor imagery tasks, after preprocessing, were used as the training set. The EEG signals from the second batch of four-category motor imagery tasks were divided into two parts according to time. The EEG signals from the first half of the second batch, after preprocessing, were used as the validation set, and the EEG signals from the second half of the second batch, after preprocessing, were used as the test set.

[0054] Step S102: Construct a motor imagery EEG signal decoding model based on brain region connectivity.

[0055] In one embodiment, such as Figure 2 As shown, the constructed EEG signal decoding model for motor imagery based on brain region connectivity includes:

[0056] The temporal module, connection relationship encoding module, spatial module, depth separable module, and prediction module are connected in sequence.

[0057] The temporal module includes: a first batch of normalized layers connected to the temporal convolutional layer; the spatial module includes: a spatial convolutional layer, a second batch of normalized layers, a first activation layer, a first average pooling layer, and a first random deactivation layer connected in sequence; the depthwise separable module includes: a depthwise separable convolutional layer, a third batch of normalized layers, a second activation layer, a second average pooling layer, and a second random deactivation layer connected in sequence.

[0058] The temporal module, the connectivity encoding module, the spatial module, the depth separability module, and the prediction module sequentially process the motor imagery task EEG signals in the EEG signal dataset to output corresponding decoding results.

[0059] In one embodiment, the prediction module employs convolutional layers to predict the category probabilities of EEG signals for different motor imagery tasks.

[0060] In one embodiment, the connectivity encoding module is used to encode the connectivity of features extracted from the EEG signal of the motor imagery task by the time-domain module to obtain brain region connectivity features;

[0061] The specific process of encoding the connection relationship includes:

[0062] Based on the electrodes used in the process of collecting EEG signals for the motor imagery task, and based on matrix calculation rules, the neighbor relationship matrix, spatial relationship matrix, routing relationship matrix, and center relationship matrix between electrodes are calculated.

[0063] The neighbor relationship matrix, spatial relationship matrix, and routing relationship matrix are added together and normalized to obtain the normalized matrix;

[0064] The features extracted by the time-domain module are multiplied by the normalized matrix, and the result of the multiplication is multiplied by the Hadamard product with the center relation matrix to obtain the brain region connectivity features.

[0065] In one embodiment, the matrix calculation rules include:

[0066] The neighbor relationship matrix calculation rules include: initializing a first all-zero connection matrix; wherein the number of rows and columns of the first all-zero connection matrix are equal to the number of electrodes; for each electrode, setting the values ​​of each of the electrode's neighboring electrodes in the corresponding positions of the first all-zero connection matrix to 1; and outputting the matrix obtained after setting the values ​​of all electrodes as the neighbor relationship matrix.

[0067] The spatial relation matrix calculation rules include: initializing a second all-zero connection matrix; wherein the number of rows and columns of the second all-zero connection matrix are equal to the number of electrodes; calculating the distance between each electrode according to the coordinates of each electrode, and filling the calculated distance between each electrode into the second all-zero connection matrix to obtain a distance matrix; quantizing the distance matrix, and performing spatial embedding encoding on the quantized distance matrix to obtain the spatial relation matrix;

[0068] The routing relation matrix calculation rules include: initializing a third all-zero connection matrix, wherein the number of rows and columns of the third all-zero connection matrix are equal to the number of electrodes; calculating the shortest path from each electrode to the other electrodes using the Floyd algorithm based on the distance matrix; encoding the shortest path of each electrode according to a set encoding structure to obtain the path features of each electrode; performing routing embedding encoding on the path features of all electrodes, and writing the result of the routing embedding encoding into the third all-zero connection matrix to obtain the routing relation matrix; wherein the set encoding structure includes: whether brain region transfer is performed, the smaller brain region number among the brain region numbers of the two electrodes constituting the shortest path, and the larger brain region number among the brain region numbers of the two electrodes constituting the shortest path;

[0069] The calculation rules for the central relation matrix include: initializing a column matrix, wherein the number of rows in the column matrix is ​​equal to the number of electrodes and the number of columns is equal to 1; calculating the degree of adjacent nodes for each electrode; performing central embedding encoding on the degree of adjacent nodes for all electrodes, and writing the result of the central embedding encoding after normalization into the column matrix to obtain the central relation matrix.

[0070] In a specific embodiment, the time-domain module can be represented by the following formula 1:

[0071] X1 = Batchnorm1(Conv1(X) in )); (Formula 1)

[0072] Among them, X in For each motor imagery task EEG signal in the EEG signal dataset, X1 is the temporal feature output by the temporal module, Conv1() represents the temporal convolutional layer, and Batchnorm1() represents the first batch of normalized layers.

[0073] The connection relationship encoding module can be represented by the following formula 2:

[0074] X2 = softmax(N) c )⊙[softmax(N n +N s +N r )X1]; (Formula 2)

[0075] Where, N n Let N be the neighbor relationship matrix. s Let N be the spatial relation matrix. r Let N be the routing relationship matrix. c y is the central relation matrix, ⊙ is the Hadamard product, and X2 is the brain region connectivity features output by the connectivity encoding module.

[0076] The spatial module can be represented by the following formula 3:

[0077] X3=Dropout1(Avgpool1(ELU1(Batchnorm2(Conv2(X2))))); (Formula 3)

[0078] Where X3 represents the features output by the spatial module, Conv2() represents the spatial convolutional layer, Batchnorm2() represents the second batch normalization layer, ELU1() represents the first activation layer, Avgpool1() represents the first average pooling layer, and Dropout1() represents the first random deactivation layer.

[0079] The depth-separable module can be represented by the following formula 4:

[0080] X4=Dropout2(Avgpool2(ELU2(Batchnorm3(SeparableConv(X3))))); (Formula 4)

[0081] Where X4 represents the features output by the depthwise separable module, SeparableConv() represents the depthwise separable convolutional layer, Batchnorm3() represents the third batch normalization layer, ELU2() represents the second activation layer, Avgpool2() represents the second average pooling layer, and Dropout2() represents the second random deactivation layer.

[0082] The prediction module can be represented by the following formula 5:

[0083] X5 = conv3(X4); (Formula 5)

[0084] Here, X5 represents the decoding result output by the prediction module, and conv3() represents the prediction module.

[0085] The following section will explain in detail the specific process of obtaining brain region connectivity features, with reference to the accompanying diagrams:

[0086] Initialize a zero-connection matrix A. Based on the positions of the 22 electrodes used during EEG acquisition, count and set the corresponding positions of the 8 neighboring electrodes of each electrode in matrix A to 1. Output the resulting matrix after setting all electrodes as a neighbor relation matrix N. n .

[0087] Specifically, Figure 3 The matrix in the diagram represents the neighbor relationship matrix between electrodes 0 to 5 within the dashed box. Each square represents the neighbor relationship between two corresponding electrodes. Black squares correspond to a value of 1, and white squares correspond to a value of 0. Taking the square in the 2nd row and 3rd column as an example, this square represents the neighbor relationship between electrode 1 and electrode 2. Taking electrode 2 as another example, its neighboring electrodes are electrodes 0, 1, 3, 4, and 5. Therefore, the values ​​in the 2nd row and 1st column, 2nd row and 2nd column, 2nd row and 4th column, 2nd row and 5th column, and 2nd row and 6th column are set to 1, while the value in the 2nd row and 3rd column remains unchanged at 0.

[0088] Furthermore, initialize an all-zero connection matrix B, calculate the distance D between each electrode according to Formula 6 below, and fill the calculated distance values ​​between each electrode into matrix B to obtain the distance matrix:

[0089]

[0090] Where x1 and y1 are the Cartesian coordinates of one electrode, and x2 and y2 are the Cartesian coordinates of the other electrode.

[0091] Furthermore, according to Formula 7 below, each distance value in the distance matrix is ​​quantized every 5 units, and the quantized distance matrix is ​​spatially embedded and encoded to obtain the spatial relation matrix N. s :

[0092] D q = 5 × (D / / 5); (Formula 7)

[0093] Where D is the distance value, D q This represents the quantized distance value, and / / indicates integer division.

[0094] For example, if a distance value is 23, the quantized distance value will be 20; if a distance value is 26, the quantized distance value will be 25.

[0095] It should be noted that quantization can reduce the calculation error of distance values ​​and reduce the amount of data, which is beneficial for subsequent processing.

[0096] The spatial embedding encoding process involves mapping identical distance values ​​in the quantized distance matrix to preset spatial relation weights to obtain the final spatial relation matrix N. s .

[0097] For example, if the quantized distance matrix contains distance values ​​of 20, 25, and 30, and the preset spatial relationship weights include a1, a2, and a3, then multiplying the quantized value of 20 by the weight of a1, the quantized value of 25 by the weight of a2, and the quantized value of 30 by the weight of a3 will result in the final spatial relationship matrix N. s .

[0098] Furthermore, an all-zero connection matrix C is initialized, and based on the distance matrix obtained in the above process, the Floyd algorithm is used to calculate the 8-adjacent shortest path between each electrode and the other electrodes.

[0099] Furthermore, the eight adjacent shortest paths of each electrode are encoded according to the structure of whether brain region transfer is performed, the smaller brain region number among the two brain region numbers of the two electrodes forming the shortest path, and the larger brain region number among the two brain region numbers of the two electrodes forming the shortest path. Specifically, the number 1 is used to indicate that brain region transfer is performed, and the number 0 is used to indicate that brain region numbering is not performed.

[0100] Specifically, such as Figure 5 As shown in the figure, the numbers in parentheses represent brain region numbers. For example, the other electrode that constitutes the 8 adjacent shortest paths of the Fz electrode is an electrode in the FC region. The smaller brain region number among the two electrodes constituting the 8 adjacent shortest paths of the Fz electrode is O, and the larger brain region number is 1. The path characteristic of the 8 adjacent shortest paths of the Fz electrode is 101.

[0101] Furthermore, route embedding encoding is performed on all the path features obtained above, and the result of route embedding encoding is written into matrix C to obtain the route relationship matrix N. r .

[0102] The route embedding encoding process involves mapping all path features to preset route relationship weights, and writing the mapping results into matrix C to obtain the route relationship matrix N. r .

[0103] Further, initialize a column matrix D. Fill in the 8-adjacency degree of each electrode from the first row to the last row according to the electrode's index. Specifically, for each electrode, the 8-adjacency degree is obtained as follows: if an electrode has n neighboring electrodes, then the 8-adjacency degree of that electrode is n; where n is an integer not greater than 8.

[0104] For example, if an electrode has 7 adjacent electrodes, then the degree of its 8 adjacent nodes is 7.

[0105] Furthermore, the 8-neighbor node degrees of all electrodes obtained above are subjected to center embedding encoding, and the result of center embedding encoding is normalized by the softmax function and written into matrix D to obtain the center relation matrix N. c .

[0106] The process of center embedding encoding involves mapping the degrees of the 8 neighboring nodes of all electrodes to preset center relation weights, and writing the mapping result into matrix D to obtain the center relation matrix N. c .

[0107] Furthermore, such as Figure 4 As shown, the neighbor relationship matrix N n Spatial relation matrix N s and routing relationship matrix N r The results are added together and then normalized using the softmax function to obtain the normalized matrix E.

[0108] Furthermore, the time-domain feature X1 output by the time-domain module is multiplied by the normalized matrix E, and the result of the multiplication is then multiplied by the central relation matrix N. c Hadamard product is performed to complete the information fusion between the temporal feature X1 output by the temporal module and the brain region connectivity.

[0109] It should be noted that those skilled in the art can initially set the spatial relationship weight, routing relationship weight, and center relationship weight according to their needs. These three weights will be iteratively updated during model training. Step S103: Using the obtained EEG signal dataset, train, validate, and test the motor imagery EEG signal decoding model so as to output the corresponding decoding result after inputting the motor imagery task EEG signal to be decoded.

[0110] In one embodiment, the training set from the EEG signal dataset is input into the motor imagery EEG signal decoding model for multiple rounds of training; the decoding results obtained after each round of training are compared with the labels in the training set, a loss function is calculated, and the loss function is backpropagated to update the model parameters of the motor imagery EEG signal decoding model; the validation set from the EEG signal dataset is used to evaluate the decoding model after each round of model parameter updates, and the decoding model with the highest validation accuracy is taken as the final motor imagery EEG signal decoding model; the test set from the EEG signal dataset is used to evaluate the final motor imagery EEG signal decoding model.

[0111] In one embodiment, the loss function Loss can be the loss function shown in Equation 8 below:

[0112]

[0113] Where N represents the number of categories of motion imagery tasks, y ic It is represented as a sign function, which is 1 when the true class of sample i is c, and 0 otherwise. X5 is the predicted class probability of each sample.

[0114] In one specific embodiment, the training process uses an adaptive moment estimation method to perform gradient descent and a cosine annealing algorithm to adjust the learning rate.

[0115] In one embodiment, the validation set is input into the model after each round of parameter updates, and the predicted class probabilities of the validation set are output. The model's validation accuracy (acc) is calculated by comparing the predicted class probabilities of the validation set with the labels of the validation set according to Equation 9 below.

[0116]

[0117] Where, N correct For the model to correctly predict the amount of data, N t The amount of data used for verification.

[0118] In one embodiment, after all rounds of training are completed, the model with the highest verification accuracy is selected as the final motor imagery EEG signal decoding model.

[0119] In one embodiment, a test set is used to evaluate the performance of the final motor imagery EEG signal decoding model.

[0120] In one specific embodiment, model training is implemented based on the PyTorch framework, using a Tesla T4 GPU to accelerate training, with a maximum training epoch of 3000 epochs, an early stopping threshold of 500 epochs, a batch size of 64, and an initial learning rate of 0.005.

[0121] In one specific embodiment, such as Figure 6As shown, the decoding method of this invention is compared with FBCSP and EEGNet. It can be seen that the average classification accuracy obtained by the decoding method of this invention is 11.57% higher than that of FBCSP and 4.82% higher than that of EEGNet.

[0122] Figure 7 This is a schematic block diagram of a motor imagery EEG signal decoding device based on brain region connectivity provided in an embodiment of this application. Figure 7 As shown, the device includes:

[0123] The acquisition and processing module 71 is used to preprocess the acquired EEG signals from multiple multi-category motor imagery tasks to obtain an EEG signal dataset.

[0124] The model building module 72 is connected to the acquisition and processing module 71 and is used to build a motor imagery EEG signal decoding model based on brain region connectivity.

[0125] The model training module 73, connected to the model building module 72, is used to train, verify and test the motor imagery EEG signal decoding model using the obtained EEG signal dataset, so as to output the corresponding decoding result after inputting the motor imagery task EEG signal to be decoded.

[0126] It should be understood that the specific process of each module performing the above-mentioned steps has been described in detail in the above method embodiments, and will not be repeated here for the sake of brevity.

[0127] It should also be understood that the module division in the embodiments of this application is illustrative and only represents a logical functional division; in actual implementation, there may be other division methods. Furthermore, the functional modules in the various embodiments of this application can be integrated into a single processor, exist as separate physical entities, or be integrated into a single module. The integrated modules described above can be implemented in hardware or as software functional modules.

[0128] In one embodiment, the specific preprocessing process includes: sequentially performing filtering, downsampling, and baseline removal processing on each acquired multi-category motor imagery task EEG signal to obtain each pre-processed EEG signal; segmenting each pre-processed EEG signal according to the corresponding motor imagery task time period to obtain multiple motor imagery task EEG signals; and labeling each motor imagery task EEG signal to obtain an EEG signal dataset.

[0129] In one embodiment, the constructed motor imagery EEG signal decoding model based on brain region connectivity includes: a temporal module, a connectivity encoding module, a spatial module, a depth-separable module, and a prediction module connected in sequence; wherein, the temporal module includes: a first batch of normalized layers connected to the temporal convolutional layers; the spatial module includes: a spatial convolutional layer, a second batch of normalized layers, a first activation layer, a first average pooling layer, and a first random deactivation layer connected in sequence; the depth-separable module includes: a depth-separable convolutional layer, a third batch of normalized layers, a second activation layer, a second average pooling layer, and a second random deactivation layer connected in sequence; the temporal module, the connectivity encoding module, the spatial module, the depth-separable module, and the prediction module sequentially process the motor imagery task EEG signals in the EEG signal dataset to output corresponding decoding results.

[0130] In one embodiment, the connectivity encoding module is used to encode the connectivity of features extracted from the EEG signals of the motor imagery task by the time-domain module to obtain brain region connectivity features. The specific process of connectivity encoding includes: calculating the neighbor relation matrix, spatial relation matrix, routing relation matrix, and central relation matrix between electrodes based on matrix calculation rules, according to the electrodes used in the process of acquiring the EEG signals of the motor imagery task; adding and normalizing the neighbor relation matrix, spatial relation matrix, and routing relation matrix to obtain a normalized matrix; multiplying the features extracted by the time-domain module with the normalized matrix, and performing a Hadamard product with the central relation matrix to obtain the brain region connectivity features.

[0131] In one embodiment, the matrix calculation rules include: a neighbor relationship matrix calculation rule, including: initializing a first all-zero connection matrix; wherein the number of rows and columns of the first all-zero connection matrix are equal to the number of electrodes; for each electrode, setting the values ​​of the corresponding positions of each of the electrode's neighboring electrodes in the first all-zero connection matrix to 1; outputting the matrix obtained after setting the values ​​of all electrodes as a neighbor relationship matrix; a spatial relationship matrix calculation rule, including: initializing a second all-zero connection matrix; wherein the number of rows and columns of the second all-zero connection matrix are equal to the number of electrodes; calculating the distance between each electrode according to the coordinates of each electrode, and filling the calculated distance between each electrode into the second all-zero connection matrix to obtain a distance matrix; quantizing the distance matrix, and performing spatial embedding encoding on the quantized distance matrix to obtain a spatial relationship matrix; a routing relationship matrix calculation rule, including: initializing a third all-zero connection matrix; wherein the third all-zero connection matrix is... The number of rows and columns in the connection matrix are equal to the number of electrodes. Based on the distance matrix, the shortest path from each electrode to the other electrodes is calculated using the Floyd algorithm. The shortest path of each electrode is encoded according to a set encoding structure to obtain the path features of each electrode. The path features of all electrodes are subjected to routing embedding encoding, and the result of the routing embedding encoding is written into the third all-zero connection matrix to obtain the routing relationship matrix. The set encoding structure includes: whether brain region transfer is performed, the smaller brain region number of the two electrodes constituting the shortest path, and the larger brain region number of the two electrodes constituting the shortest path. The central relationship matrix calculation rules include: initializing a column matrix; wherein the number of rows in the column matrix is ​​equal to the number of electrodes, and the number of columns is equal to 1; counting the degree of the adjacent nodes of each electrode; performing central embedding encoding on the degree of the adjacent nodes of all electrodes, and writing the result of the central embedding encoding after normalization into the column matrix to obtain the central relationship matrix.

[0132] In one embodiment, a motor imagery EEG signal decoding model is trained and validated using the obtained EEG signal dataset. The process includes: inputting the training set from the EEG signal dataset into the motor imagery EEG signal decoding model for multiple rounds of training; comparing the decoding results obtained after each round of training with the labels in the training set, calculating the loss function, and backpropagating the loss function to update the model parameters of the motor imagery EEG signal decoding model; evaluating the decoding model after each round of model parameter updates using the validation set from the EEG signal dataset, and using the decoding model with the highest validation accuracy as the final motor imagery EEG signal decoding model; and evaluating the final motor imagery EEG signal decoding model using the test set from the EEG signal dataset.

[0133] Figure 8 This is a schematic block diagram of a computer device / equipment / system provided in an embodiment of this application. For example... Figure 8 As shown, the computer device includes at least one processor 801, a memory 802, at least one network interface 803, and a user interface 805. The various components in the device are coupled together via a bus system 804. It is understood that the bus system 804 is used to implement communication between these components. In addition to a data bus, the bus system 804 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in… Figure 8 The general will label all buses as bus systems.

[0134] The user interface 805 may include a monitor, keyboard, mouse, trackball, clicker, button, touchpad, or touch screen.

[0135] It is understood that memory 802 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM) or programmable read-only memory (PROM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM) and synchronous static random access memory (SSRAM). The memories described in the embodiments of this invention are intended to include, but are not limited to, these and any other suitable categories of memory.

[0136] In this embodiment of the invention, the memory 802 is used to store various types of data to support the operation of the electronic terminal 800. Examples of this data include: any executable program for operation on the electronic terminal 800, such as the operating system 8021 and application programs 8022; the operating system 8021 includes various system programs, such as the framework layer, core library layer, driver layer, etc., for implementing various basic services and handling hardware-based tasks. The application program 8022 may include various applications, such as a media player, browser, etc., for implementing various application services. The method for decoding motor imagery EEG signals based on brain region connectivity provided in this embodiment of the invention can be included in the application program 8022.

[0137] The methods disclosed in the above embodiments of the present invention can be applied to processor 801, or implemented by processor 801. Processor 801 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 401 or by instructions in the form of software. The processor 801 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 801 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of the present invention. General-purpose processor 801 may be a microprocessor or any conventional processor, etc. The steps of the accessory optimization method provided in the embodiments of the present invention can be directly reflected as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software module may be located in a storage medium, which is located in a memory. The processor reads the information in the memory and combines it with its hardware to complete the steps of the aforementioned method.

[0138] In an exemplary embodiment, the electronic terminal 800 may be used by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), or complex programmable logic devices (CPLDs) to execute the aforementioned method.

[0139] According to the method provided in the embodiments of this application, this application also provides a computer program product, which includes: computer program code, which, when run on a computer, causes the computer to execute... Figure 1 The illustrated embodiment shows a method for decoding motor imagery EEG signals based on brain region connectivity.

[0140] According to the method provided in the embodiments of this application, this application also provides a computer-readable storage medium storing program code, which, when executed on a computer, causes the computer to perform... Figure 1 The embodiment shown illustrates a method for decoding motor imagery EEG signals based on brain region connectivity.

[0141] As used in this specification, the terms "component," "module," "system," etc., are used to refer to computer-related entities, hardware, firmware, combinations of hardware and software, software, or software in execution. For example, a component can be, but is not limited to, a process running on a processor, a processor, an object, an executable file, an execution thread, a program, and / or a computer. As illustrated, applications running on computing devices and computing devices can both be components. One or more components may reside in a process and / or an execution thread, and components may be located on a single computer and / or distributed among two or more computers. Furthermore, these components can be executed from various computer-readable media on which various data structures are stored. Components can communicate, for example, via local and / or remote processes based on signals having one or more data packets (e.g., data from two components interacting with another component between a local system, a distributed system, and / or a network, such as the Internet interacting with other systems via signals).

[0142] Those skilled in the art will recognize that the various illustrative logical blocks and steps described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this application.

[0143] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0144] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0145] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0146] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0147] In the above embodiments, the functions of each functional unit can be implemented entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. A computer program product includes one or more computer instructions (programs). When the computer program instructions (programs) are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. Computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., high-density digital video discs, DVDs), or semiconductor media (e.g., solid-state disks, SSDs, etc.).

[0148] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0149] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0150] In summary, this application provides a method, device, terminal, medium, and product for decoding motor imagery EEG signals based on brain region connectivity. It preprocesses multiple, multi-category motor imagery task EEG signals to obtain EEG signal datasets; constructs a motor imagery EEG signal decoding model based on brain region connectivity; and trains, validates, and tests the decoding model using the obtained EEG signal dataset to output corresponding decoding results after inputting the motor imagery task EEG signal to be decoded. This application fully considers the node relationships and dynamic topological relationships between EEG electrodes, enriching the information content of the features extracted by the model and promoting the practical application of motor imagery brain-computer interface technology. Therefore, this application effectively overcomes the various shortcomings of existing technologies and has high industrial application value.

[0151] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this application. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A method for decoding motor imagery EEG signals based on brain region connectivity, characterized in that, include: The collected EEG signals from multiple multi-category motor imagery tasks were preprocessed to obtain EEG signal datasets. Constructing a decoding model of motor imagery EEG signals based on brain region connectivity; This includes a temporal module, a connection encoding module, a spatial module, a depth-separable module, and a prediction module connected in sequence. The temporal module comprises a first batch of normalized layers connected to a temporal convolutional layer. The spatial module comprises a spatial convolutional layer, a second batch of normalized layers, a first activation layer, a first average pooling layer, and a first random deactivation layer connected in sequence. The depth-separable module comprises a depth-separable convolutional layer, a third batch of normalized layers, a second activation layer, a second average pooling layer, and a second random deactivation layer connected in sequence. The temporal module, the connection encoding module, the spatial module, the depth-separable module, and the prediction module sequentially process the EEG signals from the motor imagery task in the EEG signal dataset. The processing is performed to output the corresponding decoding results. The connectivity encoding module is used to encode the connectivity of the features extracted from the EEG signals of the motor imagery task by the time-domain module to obtain brain region connectivity features. The specific process of connectivity encoding includes: calculating the neighbor relationship matrix, spatial relationship matrix, routing relationship matrix, and central relationship matrix between electrodes based on matrix calculation rules, according to the electrodes used in the process of collecting EEG signals of the motor imagery task; adding the neighbor relationship matrix, spatial relationship matrix, and routing relationship matrix and normalizing them to obtain a normalized matrix; multiplying the features extracted by the time-domain module with the normalized matrix, and performing a Hadamard product with the central relationship matrix to obtain brain region connectivity features. The matrix calculation rules include: Neighbor relation matrix calculation rules, including: initializing a first all-zero connection matrix; wherein the number of rows and columns of the first all-zero connection matrix are equal to the number of electrodes; for each electrode, setting the values ​​of its neighboring electrodes in the first all-zero connection matrix to 1; outputting the matrix obtained after setting the values ​​of all electrodes as the neighbor relation matrix; Spatial relation matrix calculation rules, including: initializing a second all-zero connection matrix; wherein the number of rows and columns of the second all-zero connection matrix are equal to the number of electrodes; calculating the distance between each electrode according to the coordinates of each electrode, and filling the calculated distance between each electrode into the second all-zero connection matrix to obtain a distance matrix; quantizing the distance matrix, and performing spatial embedding encoding on the quantized distance matrix to obtain a spatial relation matrix; Routing relation matrix calculation rules, including: initializing a third all-zero connection matrix; wherein the third all-zero connection matrix... The number of rows and columns are equal to the number of electrodes. Based on the distance matrix, the shortest path from each electrode to the other electrodes is calculated using the Floyd algorithm. The shortest path of each electrode is encoded according to a set encoding structure to obtain the path features of each electrode. Routing embedding encoding is performed on the path features of all electrodes, and the result of the routing embedding encoding is written into the third all-zero connection matrix to obtain the routing relation matrix. The set encoding structure includes: whether brain region transfer is performed, the smaller brain region number among the two brain region numbers of the two electrodes forming the shortest path, and the larger brain region number among the two brain region numbers of the two electrodes forming the shortest path. The central relation matrix calculation rules include: initializing a column matrix; where the number of rows in the column matrix is ​​equal to the number of electrodes, and the number of columns is equal to 1; calculating the degree of adjacent nodes for each electrode; performing central embedding encoding on the degree of adjacent nodes for all electrodes, and writing the result of the central embedding encoding after normalization into the column matrix to obtain the central relation matrix. Using the obtained EEG signal dataset, a motor imagery EEG signal decoding model was trained, validated, and tested to output the corresponding decoding result after inputting the motor imagery task EEG signal to be decoded.

2. The method for decoding motor imagery EEG signals based on brain region connectivity as described in claim 1, characterized in that, The specific preprocessing steps include: Each acquired EEG signal for a multi-category motor imagery task was sequentially filtered, downsampled, and baseline removed to obtain each pre-processed EEG signal. Each pre-processed EEG signal was segmented according to the corresponding time period of the motor imagery task to obtain multiple motor imagery task EEG signals. The EEG signals for each motor imagery task were labeled to obtain an EEG signal dataset.

3. The method for decoding motor imagery EEG signals based on brain region connectivity as described in claim 1, characterized in that, Using the obtained EEG signal dataset, a motor imagery EEG signal decoding model is trained and validated. The process includes: The training set from the EEG signal dataset is input into the motor imagery EEG signal decoding model for multiple rounds of training. The decoding results obtained after each round of training are compared with the labels in the training set, the loss function is calculated and backpropagated to update the model parameters of the motor imagery EEG signal decoding model. The validation set in the EEG signal dataset is used to evaluate the decoding model after each round of model parameter updates, and the decoding model with the highest validation accuracy is used as the final motor imagery EEG signal decoding model. The final motor imagery EEG signal decoding model was evaluated using a test set from the EEG signal dataset.

4. A device for decoding motor imagery EEG signals based on brain region connectivity, characterized in that, include: The acquisition and processing module is used to preprocess the acquired EEG signals from multiple multi-category motor imagery tasks to obtain an EEG signal dataset. The model building module, connected to the acquisition and processing module, is used to build a motor imagery EEG signal decoding model based on brain region connectivity. This includes a temporal module, a connection encoding module, a spatial module, a depth-separable module, and a prediction module connected in sequence. The temporal module comprises a first batch of normalized layers connected to a temporal convolutional layer. The spatial module comprises a spatial convolutional layer, a second batch of normalized layers, a first activation layer, a first average pooling layer, and a first random deactivation layer connected in sequence. The depth-separable module comprises a depth-separable convolutional layer, a third batch of normalized layers, a second activation layer, a second average pooling layer, and a second random deactivation layer connected in sequence. The temporal module, the connection encoding module, the spatial module, the depth-separable module, and the prediction module sequentially process the EEG signals from the motor imagery task in the EEG signal dataset. The processing is performed to output the corresponding decoding results. The connectivity encoding module is used to encode the connectivity of the features extracted from the EEG signals of the motor imagery task by the time-domain module to obtain brain region connectivity features. The specific process of connectivity encoding includes: calculating the neighbor relationship matrix, spatial relationship matrix, routing relationship matrix, and central relationship matrix between electrodes based on matrix calculation rules, according to the electrodes used in the process of collecting EEG signals of the motor imagery task; adding the neighbor relationship matrix, spatial relationship matrix, and routing relationship matrix and normalizing them to obtain a normalized matrix; multiplying the features extracted by the time-domain module with the normalized matrix, and performing a Hadamard product with the central relationship matrix to obtain brain region connectivity features. The matrix calculation rules include: Neighbor relation matrix calculation rules, including: initializing a first all-zero connection matrix; wherein the number of rows and columns of the first all-zero connection matrix are equal to the number of electrodes; for each electrode, setting the values ​​of its neighboring electrodes in the first all-zero connection matrix to 1; outputting the matrix obtained after setting the values ​​of all electrodes as the neighbor relation matrix; Spatial relation matrix calculation rules, including: initializing a second all-zero connection matrix; wherein the number of rows and columns of the second all-zero connection matrix are equal to the number of electrodes; calculating the distance between each electrode according to the coordinates of each electrode, and filling the calculated distance between each electrode into the second all-zero connection matrix to obtain a distance matrix; quantizing the distance matrix, and performing spatial embedding encoding on the quantized distance matrix to obtain a spatial relation matrix; Routing relation matrix calculation rules, including: initializing a third all-zero connection matrix; wherein the third all-zero connection matrix... The number of rows and columns are equal to the number of electrodes. Based on the distance matrix, the shortest path from each electrode to the other electrodes is calculated using the Floyd algorithm. The shortest path of each electrode is encoded according to a set encoding structure to obtain the path features of each electrode. Routing embedding encoding is performed on the path features of all electrodes, and the result of the routing embedding encoding is written into the third all-zero connection matrix to obtain the routing relation matrix. The set encoding structure includes: whether brain region transfer is performed, the smaller brain region number among the two brain region numbers of the two electrodes forming the shortest path, and the larger brain region number among the two brain region numbers of the two electrodes forming the shortest path. The central relation matrix calculation rules include: initializing a column matrix; where the number of rows in the column matrix is ​​equal to the number of electrodes, and the number of columns is equal to 1; calculating the degree of adjacent nodes for each electrode; performing central embedding encoding on the degree of adjacent nodes for all electrodes, and writing the result of the central embedding encoding after normalization into the column matrix to obtain the central relation matrix. The model training module, connected to the model building module, is used to train, verify, and test the motor imagery EEG signal decoding model using the obtained EEG signal dataset, so as to output the corresponding decoding result after inputting the motor imagery task EEG signal to be decoded.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 3.

6. A computer program product, characterized in that, The computer program product includes computer program code that, when run on a computer, causes the computer to implement the method as described in any one of claims 1 to 3.

7. An electronic terminal, comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1 to 3.

Images