P300 detection method and device based on double-flow space-time diagram convolutional neural network
Through the detection method of dual-stream spatiotemporal graph convolution neural network, the problem of insufficient accuracy and efficiency of P300 event-related potential detection in the existing technology is solved, and more efficient EEG signal feature extraction and fusion is achieved, thereby improving the brain-computer interface detection performance.
Patent Information
- Application Number
- CN202411174691.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-26
- Publication Date
- 2024-12-17
AI Technical Summary
The existing P300 event-related potential detection methods have insufficient accuracy and efficiency, and it is difficult to effectively capture complex spatio-temporal information in EEG signals, affecting the application effect of brain-computer interfaces.
The detection method based on the dual-stream spatiotemporal graph convolution neural network is adopted, and the space-time mapping module, the double-stream convolution spatiotemporal feature extraction module, the attention spatiotemporal feature fusion module and the result prediction module are combined with one-dimensional convolution unit, subspace mapping layer, The batch normalization layer, graph convolution layer and attention layer extract and fuse the spatial and temporal characteristics of the EEG signal, and output the detection results of the P300 event-related potential.
It improves the detection accuracy and recognition efficiency of P300 event-related potentials, can capture complex complementary information in EEG signals more efficiently and accurately, and improves the detection performance of the brain-computer interface.
Smart Images
Figure CN119138910A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of potential detection technology, and in particular to a P300 detection method and device based on a dual-stream spatiotemporal graph convolutional neural network. Background Art
[0002] The brain-computer interface (BCI) establishes a direct path between the human brain and external devices, allowing people to communicate directly with the outside world through brain activity without any action, improving the quality of life of those who cannot use traditional input devices due to muscle weakness or other movement disorders. Event-related potential (ERP) signals are widely used in the field of brain-computer interfaces; among the many types of ERP signals, the P300 event-related potential represents the positive peak that appears about 300 milliseconds after a person receives a target stimulus, and is the most widely used in brain-computer interfaces. Researchers point out that when a person observes a series of visual stimuli, the P300 event-related potential is induced when the target object is observed. Therefore, accurately identifying the P300 evoked potential in the EEG signal is an important key technology in the field of BCI. Summary of the invention
[0003] In order to solve the deficiencies mentioned in the above background technology, the purpose of the present invention is to provide a P300 detection method and device based on a dual-stream spatiotemporal graph convolutional neural network.
[0004] In a first aspect, the purpose of the present invention can be achieved by the following technical solution: a P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network, the method comprising the following steps:
[0005] Acquire a number of P300 event-related potential EEG data and corresponding P300 event-related potential labels, pre-process the P300 event-related potential EEG data, and obtain processed P300 event-related potential EEG data;
[0006] Input the processed P300 event-related potential EEG data and the corresponding P300 event-related potential label into a pre-established two-stream spatiotemporal graph convolutional neural network model for training, thereby obtaining a trained two-stream spatiotemporal graph convolutional neural network model;
[0007] Obtain the EEG samples to be identified, input the EEG samples to be identified into the trained two-stream spatiotemporal graph convolutional neural network model, and output the P300 event-related potential detection result.
[0008] In combination with the first aspect, in some implementations of the first aspect, the method further includes: Represents a certain EEG data, C is the number of EEG electrodes, and N is the number of time points;
[0009] In the corresponding P300 event-related potential label: 0 indicates that the EEG sample does not contain the P300 event-related potential, and 1 indicates that the EEG sample contains the P300 event-related potential.
[0010] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the process of preprocessing the plurality of P300 event-related potential EEG data includes filtering and standardization, wherein the plurality of P300 event-related potential EEG data are filtered using a bandpass filter, wherein the bandpass filter is a 4th-order Butterworth filter;
[0011] The EEG data after bandpass filtering is standardized to obtain the processed P300 event-related potential EEG data:
[0012]
[0013] Where X ij is the jth time point of the ith electrode in sample X, and σ i is the mean and standard deviation of the ith electrode.
[0014] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the pre-established two-stream spatiotemporal graph convolutional neural network model includes: a spatiotemporal mapping module, a two-stream graph convolutional spatiotemporal feature extraction module, an attention spatiotemporal feature fusion module, and a result prediction module;
[0015] The spatiotemporal mapping module is used to transform the input EEG data dimension into Dynamically learn the mutual coupling between the temporal and spatial dimensions of EEG signals, where V S =V T =C, the dual-stream graph convolution spatiotemporal feature extraction module is used to extract the spatial and temporal information in the EEG data respectively, and the attention spatiotemporal feature fusion module is based on the attention mechanism to fuse the temporal and spatial features extracted by the two branches in the dual-stream graph convolution spatiotemporal feature extraction module to obtain high-dimensional spatiotemporal features. The result prediction module is used to obtain the high-dimensional spatiotemporal features and output the prediction result of whether the P300 event-related potential is included as the P300 event-related potential detection result.
[0016] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the spatiotemporal mapping module is composed of a one-dimensional convolution unit and a subspace mapping layer;
[0017] Transform the input EEG data dimension into The calculation formula is:
[0018] X M =W T X CNN W S +B ST
[0019] in represents the output of a one-dimensional convolutional unit, and are trainable parameters.
[0020] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the dual-stream graph convolution spatiotemporal feature extraction module includes two parallel branches, a spatial feature extraction branch and a temporal feature extraction branch, which respectively extract spatial and temporal features from the EEG signal, wherein the spatial feature extraction branch is composed of a batch normalization layer, a spatial graph convolution layer, and a spatial feature attention layer, and selects X M Each row in is used as a node of the spatial graph convolution layer; the temporal feature extraction branch consists of a batch normalization layer, a temporal graph convolution layer, and a temporal feature attention layer. M Each column in serves as a node in the temporal graph convolutional layer.
[0021] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the adjacency matrix of the spatial graph convolution layer of the spatial feature extraction branch is dynamically updated during the training process, and the calculation method is:
[0022] A S =Softmax(h(X M )h(X M ) T )
[0023] Where h( ) represents the tanh activation function; the spatial graph convolution layer uses a K-order polynomial graph filter to perform graph convolution in the spatial perspective, and the calculation method is:
[0024]
[0025] Among them, α k are the polynomial coefficients, is the symmetric normalized Laplacian matrix of the spatial graph, is a K-order filter, is the dimension transformation matrix, and the spatial feature attention layer is the feature output by the spatial graph convolution layer. The channel adaptively allocates weights to extract task-related spatial features from highly redundant EEG signals. The calculation method is:
[0026]
[0027] in and G S The output after average pooling and maximum pooling, is a trainable parameter, r is the dimension reduction ratio;
[0028] The adjacency matrix of the temporal graph convolution layer in the temporal feature extraction branch is dynamically updated during the training process and is calculated as follows:
[0029] A T =Softmax(h(X M ) T h(X M ))
[0030] Where h() represents the tanh activation function. The temporal graph convolution layer uses a K-order polynomial graph filter to perform graph convolution in the temporal perspective. The calculation method is:
[0031]
[0032] Among them, β k are the polynomial coefficients, is the symmetric normalized Laplace matrix of the time graph, is a K-order filter, is the dimension transformation matrix, and the temporal feature attention layer is the feature output by the temporal graph convolution layer. The channel adaptively allocates weights to extract task-related time features from highly redundant EEG signals. The calculation method is:
[0033]
[0034] in and G T The output after average pooling and maximum pooling, is a trainable parameter and r is the dimension reduction ratio.
[0035] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the calculation method of the attention spatiotemporal feature fusion module is:
[0036]
[0037] F ST =O×V T +V S
[0038] Where Q S=F S W AS ,and is a trainable parameter, d = out; K T =F T W AT ,and is a trainable parameter, The calculated attention score matrix, is the final space-time feature, as a high-dimensional space-time feature;
[0039] The result prediction module consists of a first fully connected layer and a second fully connected layer.
[0040] In combination with the first aspect, in some implementations of the first aspect, the method further includes: the loss function used in the training process of the pre-established two-stream spatiotemporal graph convolutional neural network model is a standard cross entropy loss function, and the calculation method is:
[0041]
[0042] Where M is the total number of EEG samples used for training, y ic is the true label of the P300 event-related potential corresponding to the sample, It is the prediction result of the two-stream spatiotemporal graph convolutional neural network model.
[0043] In a second aspect, in order to achieve the above-mentioned object, the present invention discloses a P300 detection device based on a dual-stream spatiotemporal graph convolutional neural network, comprising:
[0044] A data processing module is used to obtain a number of P300 event-related potential EEG data and corresponding P300 event-related potential labels, pre-process the P300 event-related potential EEG data, and obtain processed P300 event-related potential EEG data;
[0045] A model training module is used to input the processed P300 event-related potential EEG data and the corresponding P300 event-related potential label into a pre-established two-stream spatiotemporal graph convolutional neural network model for training, thereby obtaining a trained two-stream spatiotemporal graph convolutional neural network model;
[0046] The potential detection module is used to obtain the EEG samples to be identified, input the EEG samples to be identified into the trained dual-stream spatiotemporal graph convolutional neural network model, and output the P300 event-related potential detection results.
[0047] Beneficial effects of the present invention:
[0048] The present invention is based on graph neural networks, which can more efficiently and accurately capture the complex and complementary time and space information in EEG signals, and use the attention mechanism to fuse the time and space information to obtain high-dimensional space-time features; the present invention has a higher recognition accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0049] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0050] Figure 1 It is a schematic flow chart of the method of the present invention;
[0051] Figure 2 It is a schematic diagram of the architecture of the dual-stream spatiotemporal graph convolutional neural network of the present invention;
[0052] Figure 3 It is a schematic diagram of the structure of the device of the present invention. DETAILED DESCRIPTION
[0053] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0054] Embodiment 1:
[0055] The following is an introduction to the relevant terms involved in the embodiments of the present application:
[0056] Convolutional Neural Networks (CNN) are a type of feedforward neural network that includes convolutional calculations and has a deep structure. It is one of the representative algorithms of deep learning. Convolutional neural networks have representation learning capabilities and can perform shift-invariant classification on input information according to their hierarchical structure. Therefore, they are also called "Shift-Invariant Artificial Neural Networks (SIANN)".
[0057] P300 potential: P300 potential is a type of ERP (event-related potential), which is an endogenous special evoked potential related to cognitive function.
[0058] like Figure 1 As shown, a P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network includes the following steps:
[0059] Acquire a number of P300 event-related potential EEG data and corresponding P300 event-related potential labels, pre-process the P300 event-related potential EEG data, and obtain processed P300 event-related potential EEG data;
[0060] Obtain some P300 event-related potential EEG data, including Represents a certain EEG data, C is the number of EEG electrodes, and N is the number of time points;
[0061] Get the labels corresponding to several P300 event-related potential EEG data, 0 means that the EEG sample does not contain the P300 event-related potential, and 1 means that the EEG sample contains the P300 event-related potential.
[0062] The acquired EEG samples are filtered using a 0.1-20 Hz bandpass filter, where the bandpass filter is a 4th-order Butterworth filter;
[0063] Standardize the EEG data after bandpass filtering:
[0064]
[0065] Where X ij is the jth time point of the ith electrode in sample X, and σ i is the mean and standard deviation of the ith electrode.
[0066] Input the processed P300 event-related potential EEG data and the corresponding P300 event-related potential label into a pre-established two-stream spatiotemporal graph convolutional neural network model for training, thereby obtaining a trained two-stream spatiotemporal graph convolutional neural network model;
[0067] The pre-established two-stream spatiotemporal graph convolutional neural network model includes: spatiotemporal mapping module, two-stream graph convolution spatiotemporal feature extraction module, attention spatiotemporal feature fusion module and result prediction module;
[0068] The spatiotemporal mapping module consists of a one-dimensional convolution unit and a subspace mapping layer, which transforms the input EEG data into Where V S =V T= C. The one-dimensional convolution unit contains 3 convolution layers, the convolution kernel size is 3, the step size is 2, and the padding is 1. The subspace mapping layer further transforms the output of the one-dimensional convolution unit: X M =W T X CNN W S +B ST .in represents the output of a one-dimensional convolutional unit, and is a trainable parameter.
[0069] The dual-stream graph convolution spatiotemporal feature extraction module contains two parallel branches, which extract spatial and temporal features from EEG signals respectively. The spatial feature extraction branch consists of a batch normalization layer, a spatial graph convolution layer, and a spatial feature attention layer. M Each row in is used as a node of the spatial graph convolution layer; the temporal feature extraction branch consists of a batch normalization layer, a temporal graph convolution layer, and a temporal feature attention layer. M Each column in is used as a node of the temporal graph convolution layer. The adjacency matrix of the spatial graph convolution layer is dynamically updated during the training process, and its calculation method is:
[0070] A S =Softmax(h(X M )h(X M ) T ).
[0071] Where h( ) represents the tanh activation function. In order to ensure the sparsity of the adjacency matrix, in A S In each row of , the first 50% elements with the largest value are retained, and the remaining elements are set to 0. The spatial graph convolution layer uses a graph filter in the form of a K-order polynomial to perform graph convolution in the spatial perspective to complete the update of the spatial node feature information. The calculation method is:
[0072]
[0073] The spatial feature attention layer is the feature output by the spatial graph convolution layer. The channel adaptively allocates weights to extract task-related spatial features from highly redundant EEG signals. The calculation method is: The adjacency matrix of the temporal graph convolution layer is also dynamically updated, and its calculation method is: T =Softmax(h(X M ) T h(X M )). For A T , similarly only retaining the top 50% of the elements with the largest values in each row.
[0074] The temporal graph convolution is similar to the spatial graph convolution, which completes the update of the feature information of the time node. Its calculation method is: The structure of the temporal feature attention layer is the same as that of the spatial feature attention layer, with the input being G T , output the time feature F after the feature channel weighting T , which is calculated as
[0075] The attention spatiotemporal feature fusion module uses the attention mechanism to adaptively fuse the temporal features and spatial features extracted by the dual-stream graph convolution spatiotemporal feature extraction module to obtain the final high-dimensional spatiotemporal features. The specific calculation method is:
[0076]
[0077] F ST =O×V T +V S
[0078] Where Q S =F S W AS ,and is a trainable parameter, d = out = 64; similarly, K T =F T W AT ,and is a trainable parameter. The calculated attention score matrix, It is the final space-time feature.
[0079] The result prediction module consists of the first fully connected layer and the second fully connected layer. The input dimension of the first fully connected layer is V S ×d, the output dimension is 64; the input dimension of the second fully connected layer is 64, and the output dimension is 2.
[0080] Obtain the EEG samples to be identified, input the EEG samples to be identified into the trained two-stream spatiotemporal graph convolutional neural network model, and output the P300 event-related potential detection result.
[0081] The loss function used to train the two-stream spatiotemporal graph convolutional neural network is the standard cross entropy loss function, which is calculated as: Where M is the total number of EEG samples used for training, y ic is the true label of the P300 event-related potential corresponding to the sample, It is the prediction result of the two-stream spatiotemporal graph convolutional neural network.
[0082] Specifically, the present invention is further described below by way of embodiments:
[0083] In order to verify the effectiveness of the proposed P300 event-related potential detection method based on two-stream spatiotemporal graph convolutional neural network, the widely used Dataset II dataset in BCI competition III (including two subjects A and B) was selected for simulation experiments, and several excellent P300 event-related potential detection methods were selected for comparison. The selected comparison methods include: spatiotemporal attention capsule network (ST-CapsNet), event-related potential capsule network (ERP-CapsNet), convolutional neural network (CNN-1), integrated convolutional neural network (MCNN-1), linear discriminability analysis (LDA), spatial weighted Fisher principal component analysis (SWFP), and multi-scale convolutional neural network (MsCNN). In the simulation experiment, accuracy and F1-score were selected as performance measurement indicators, and the results are shown in Table 1.
[0084] Table 1 Experimental results of the dual-stream spatiotemporal graph convolutional neural network in the P300 event-related potential detection method in the simulation experiment
[0085]
[0086] From the results, it can be seen that the P300 event-related potential detection method based on the dual-stream spatiotemporal graph convolutional neural network proposed in the present invention has achieved the highest accuracy and F1 score compared with other methods with excellent performance. Based on the graph convolutional neural network, the present invention can more effectively capture the spatiotemporal information in EEG signals and improve the detection performance of the P300 event-related potential.
[0087] Embodiment 2: In the second aspect, as Figure 3 As shown, in order to achieve the above purpose, the present invention discloses a P300 detection device based on a dual-stream spatiotemporal graph convolutional neural network, comprising:
[0088] The data processing module 11 is used to obtain a number of P300 event-related potential EEG data and corresponding P300 event-related potential labels, and pre-process the P300 event-related potential EEG data to obtain processed P300 event-related potential EEG data;
[0089] The model training module 12 is used to input the processed P300 event-related potential EEG data and the corresponding P300 event-related potential label into a pre-established dual-stream spatiotemporal graph convolutional neural network model for training, thereby obtaining a trained dual-stream spatiotemporal graph convolutional neural network model;
[0090] The potential detection module 13 is used to obtain the EEG samples to be identified, input the EEG samples to be identified into the trained dual-stream spatiotemporal graph convolutional neural network model, and output the P300 event-related potential detection result.
[0091] Based on the same inventive concept, the present invention also provides a computer device, which includes: one or more processors, and a memory for storing one or more computer programs; the program includes program instructions, and the processor is used to execute the program instructions stored in the memory. The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is used to implement one or more instructions, specifically for loading and executing one or more instructions in a computer storage medium to implement the above method.
[0092] It needs to be further explained that, based on the same inventive concept, the present invention also provides a computer storage medium, on which a computer program is stored, and the computer program is executed by a processor to execute the above method. The storage medium can adopt any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electrical, magnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.
[0093] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present disclosure. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.
[0094] The above shows and describes the basic principles, main features and advantages of the present disclosure. Those skilled in the art should understand that the present disclosure is not limited by the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present disclosure. Without departing from the spirit and scope of the present disclosure, the present disclosure may have various changes and improvements, and these changes and improvements fall within the scope of the present disclosure to be protected.
Claims
1. A P300 detection method based on a two-stream spatiotemporal graph convolutional neural network, characterized in that: The method comprises the following steps: Acquire a number of P300 event-related potential EEG data and corresponding P300 event-related potential labels, pre-process the P300 event-related potential EEG data, and obtain processed P300 event-related potential EEG data; Input the processed P300 event-related potential EEG data and the corresponding P300 event-related potential label into a pre-established two-stream spatiotemporal graph convolutional neural network model for training, thereby obtaining a trained two-stream spatiotemporal graph convolutional neural network model; Obtain the EEG samples to be identified, input the EEG samples to be identified into the trained two-stream spatiotemporal graph convolutional neural network model, and output the P300 event-related potential detection result.
2. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 1 is characterized in that: The P300 event-related potential EEG data Represents a certain EEG data, C is the number of EEG electrodes, and N is the number of time points; In the corresponding P300 event-related potential label: 0 indicates that the EEG sample does not contain the P300 event-related potential, and 1 indicates that the EEG sample contains the P300 event-related potential.
3. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 1 is characterized in that: The process of preprocessing the plurality of P300 event-related potential EEG data includes filtering and standardization, wherein the plurality of P300 event-related potential EEG data are filtered using a bandpass filter, wherein the bandpass filter is a 4th-order Butterworth filter; The EEG data after bandpass filtering is standardized to obtain the processed P300 event-related potential EEG data: Where X ij is the jth time point of the ith electrode in sample X, and σ i is the mean and standard deviation of the ith electrode.
4. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 1 is characterized in that: The pre-established dual-stream spatiotemporal graph convolutional neural network model includes: a spatiotemporal mapping module, a dual-stream graph convolutional spatiotemporal feature extraction module, an attention spatiotemporal feature fusion module and a result prediction module; The spatiotemporal mapping module is used to transform the input EEG data dimension into Dynamically learn the mutual coupling between the temporal and spatial dimensions of EEG signals, where V S =V T =C, the dual-stream graph convolution spatiotemporal feature extraction module is used to extract the spatial and temporal information in the EEG data respectively, and the attention spatiotemporal feature fusion module is based on the attention mechanism to fuse the temporal and spatial features extracted by the two branches in the dual-stream graph convolution spatiotemporal feature extraction module to obtain high-dimensional spatiotemporal features. The result prediction module is used to obtain the high-dimensional spatiotemporal features and output the prediction result of whether the P300 event-related potential is included as the P300 event-related potential detection result.
5. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 4 is characterized in that: The spatiotemporal mapping module consists of a one-dimensional convolution unit and a subspace mapping layer; Transform the input EEG data dimension into The calculation formula is: X M =W T X CNN W S +B ST in represents the output of a one-dimensional convolutional unit, and are trainable parameters.
6. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 4 is characterized in that: The dual-stream graph convolution spatiotemporal feature extraction module contains two parallel branches, a spatial feature extraction branch and a temporal feature extraction branch, which respectively extract spatial and temporal features from EEG signals. The spatial feature extraction branch consists of a batch normalization layer, a spatial graph convolution layer, and a spatial feature attention layer. M Each row in is used as a node of the spatial graph convolution layer; The temporal feature extraction branch consists of a batch normalization layer, a temporal graph convolution layer, and a temporal feature attention layer. M Each column in serves as a node in the temporal graph convolutional layer.
7. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 6 is characterized in that: The adjacency matrix of the spatial graph convolution layer of the spatial feature extraction branch is dynamically updated during the training process and is calculated as follows: A S =Softmax(h(X M )h(X M ) T ) Where h( ) represents the tanh activation function; the spatial graph convolution layer uses a K-order polynomial graph filter to perform graph convolution in the spatial perspective, and the calculation method is: Among them, α k are the polynomial coefficients, is the symmetric normalized Laplacian matrix of the spatial graph, is a K-order filter, is the dimension transformation matrix, and the spatial feature attention layer is the feature output by the spatial graph convolution layer. The channel adaptively allocates weights to extract task-related spatial features from highly redundant EEG signals. The calculation method is: in and G S The output after average pooling and maximum pooling, is a trainable parameter, r is the dimension reduction ratio; The adjacency matrix of the temporal graph convolution layer in the temporal feature extraction branch is dynamically updated during the training process and is calculated as follows: A T =Softmax(h(X M ) T h(X M )) Where h() represents the tanh activation function. The temporal graph convolution layer uses a K-order polynomial graph filter to perform graph convolution in the temporal perspective. The calculation method is: Among them, β k are the polynomial coefficients, is the symmetric normalized Laplace matrix of the time graph, is a K-order filter, is the dimension transformation matrix, and the temporal feature attention layer is the feature output by the temporal graph convolution layer. The channel adaptively allocates weights to extract task-related time features from highly redundant EEG signals. The calculation method is: in and G T The output after average pooling and maximum pooling, is a trainable parameter and r is the dimension reduction ratio.
8. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 4 is characterized in that: The calculation method of the attention spatiotemporal feature fusion module is: F ST =O×V T +V S Where Q S =F S W AS ,and is a trainable parameter, d = out; K T =F T W AT ,and is a trainable parameter, The calculated attention score matrix, is the final space-time feature, as a high-dimensional space-time feature; The result prediction module consists of a first fully connected layer and a second fully connected layer.
9. The P300 detection method based on a dual-stream spatiotemporal graph convolutional neural network according to claim 8, characterized in that: The loss function used in the training process of the pre-established two-stream spatiotemporal graph convolutional neural network model is the standard cross entropy loss function, which is calculated as follows: Where M is the total number of EEG samples used for training, y ic is the true label of the P300 event-related potential corresponding to the sample, It is the prediction result of the two-stream spatiotemporal graph convolutional neural network model.
10. A P300 detection device based on a dual-stream spatiotemporal graph convolutional neural network, characterized in that: include: A data processing module is used to obtain a number of P300 event-related potential EEG data and corresponding P300 event-related potential labels, pre-process the P300 event-related potential EEG data, and obtain processed P300 event-related potential EEG data; A model training module is used to input the processed P300 event-related potential EEG data and the corresponding P300 event-related potential label into a pre-established two-stream spatiotemporal graph convolutional neural network model for training, thereby obtaining a trained two-stream spatiotemporal graph convolutional neural network model; The potential detection module is used to obtain the EEG samples to be identified, input the EEG samples to be identified into the trained dual-stream spatiotemporal graph convolutional neural network model, and output the P300 event-related potential detection results.