An ERP signal detection method, system, computer device, and computer-readable storage medium

Abstract

The present invention discloses an ERP signal detection method, system, computer device, and computer-readable storage medium. The method constructs a frequency-space-time framework using a frequency feature encoder, a spatial feature encoder, and a temporal feature encoder to encode frequency, spatial, and temporal features respectively, and fuses them to obtain global features, and then classifies according to the global features. The detection method of the present invention can fully extract the global features in the EEG signals while considering the characteristics of EEG signals and ERP signals, so as to learn the potentially invariant EEG signal features among different subjects, and has high detection accuracy and cross-subject robustness in the cross-subject ERP signal detection task. In addition, compared with traditional machine learning methods, the present invention does not need to manually extract complex relevant features, thereby retaining more relevant information in the EEG signals, and achieving a higher-efficiency and high-precision ERP detection task through end-to-end training.

Description

Technical Field

[0001] The present invention relates to the field of information technology, and in particular to an ERP signal detection method and system, a computer device, and a computer-readable storage medium. Background Art

[0002] A brain-computer interface (BCI) system provides a communication bridge between the human brain and external devices by decoding the brain activities of users. Due to the advantages of electroencephalogram (EEG) in terms of non-invasiveness, zero risk, and relative portability, these systems are usually developed using EEG. Event-related potential (ERP) signals based on electroencephalogram can identify high-level cognitive activities in the human brain stimulated by specific events, which play an important role in many applications, such as object detection and medical diagnosis. Therefore, in the related research of BCI systems, the ERP detection task has received more and more attention. However, due to the characteristics of electroencephalogram signals, such as low signal-to-noise ratio and large cross-subject variability, it is still challenging to develop a high-precision ERP detection method without pre-calibration.

[0003] There have been many traditional methods to address the above challenges, such as linear discriminant analysis (LDA), sequential linear discriminant analysis (SWLDA), support vector machine (SVM), logistic regression (LR), etc. However, the detection performance of these methods usually depends on handcrafted features, so the characterization ability is limited and the detection level is low. On the other hand, current deep learning methods have made great progress in the field of EEG signal analysis. This method can adaptively mine high-level features driven by data, avoiding manual feature selection and feature dimensionality reduction, thereby improving the performance of the detection task. Convolutional neural network (CNN) can effectively extract local data features, and long short-term memory network (LSTM) can effectively extract temporal signal features. However, the above methods still have a relatively shallow exploration and utilization level of the EEG data itself. Therefore, how to fully extract its global representation according to the characteristics of EEG signals to obtain an ERP detection method with strong anti-noise ability, high robustness, and high detection accuracy is still a challenge at present. Summary of the Invention

[0004] The present invention provides an ERP signal detection method and system, a computer device, and a computer-readable storage medium, which are used to overcome the defects such as low robustness and detection accuracy in the prior art.

[0005] To achieve the above object, the present invention proposes an ERP signal detection method, including the following steps:

[0006] Obtain RSVP EEG data, preprocess the RSVP EEG data to obtain an RSVP reference data set;

[0007] Obtain the EEG node connection relationship according to the adjacent relationship of the electrodes on the EEG cap;

[0008] Construct an ERP signal detection model; the ERP signal detection model includes a frequency feature encoder, a spatial feature encoder, and a temporal feature encoder connected in series;

[0009] Use the RSVP benchmark dataset and the EEG node connection relationship to train and validate the ERP signal detection model in a cross-subject cross-validation manner to determine the optimal parameters of the ERP signal detection model;

[0010] Input the preprocessed RSVP EEG data to be measured into the trained ERP signal detection model, use the frequency feature encoder, the spatial feature encoder, and the temporal feature encoder to encode and fuse the frequency, spatial, and temporal features respectively to obtain global features, and perform binary classification according to the global features to obtain a classification result, and judge whether there is an ERP signal in the RSVP EEG data to be measured.

[0011] To achieve the above object, the present invention also proposes an ERP signal detection system, including:

[0012] A data acquisition module, configured to acquire RSVP EEG data, preprocess the RSVP EEG data to obtain an RSVP benchmark dataset; obtain the EEG node connection relationship according to the adjacent relationship of the electrodes on the EEG cap;

[0013] A model construction module, configured to construct an ERP signal detection model; the ERP signal detection model includes a frequency feature encoder, a spatial feature encoder, and a temporal feature encoder connected in series; use the RSVP benchmark dataset and the EEG node connection relationship to train and validate the ERP signal detection model in a cross-subject cross-validation manner to determine the optimal parameters of the ERP signal detection model;

[0014] A detection module, configured to input the preprocessed RSVP EEG data to be measured into the trained ERP signal detection model, use the frequency feature encoder, the spatial feature encoder, and the temporal feature encoder to encode and fuse the frequency, spatial, and temporal features respectively to obtain global features, and perform binary classification according to the global features to obtain a classification result, and judge whether there is an ERP signal in the RSVP EEG data to be measured.

[0015] To achieve the above object, the present invention also proposes a computer device, including a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the steps of the above method are implemented.

[0016] To achieve the above object, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0018] The ERP signal detection method provided by the present invention constructs a frequency-space-time framework by using a frequency feature encoder, a spatial feature encoder, and a temporal feature encoder, encodes the frequency, spatial, and temporal features respectively, and fuses them to obtain global features, and then classifies according to the global features. The detection method of the present invention can fully extract the global features in the EEG signals while considering the characteristics of EEG signals and ERP signals, so as to learn the potentially invariant EEG signal features among different subjects, and has high detection accuracy and cross-subject robustness in the cross-subject ERP signal detection task. In addition, compared with traditional machine learning methods, the present invention does not need to manually extract complex relevant features, so as to retain more relevant information in the EEG signals, and realizes a higher-efficiency and high-precision ERP detection task through end-to-end training. Description of the Drawings

[0019] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on the structures shown in these drawings without creative efforts.

[0020] Figure 1 It is a flowchart of the ERP signal detection method provided by the present invention;

[0021] Figure 2 It is an overall framework diagram of the ERP signal detection method provided by the present invention;

[0022] Figure 3 It is an algorithm block diagram of the ERP signal detection model designed in the embodiments of the present invention;

[0023] Figure 4 It is a flowchart of the RSVP experimental paradigm adopted by the public dataset in the embodiments of the present invention;

[0024] Figure 5 It is a schematic diagram of node connection in the embodiments of the present invention;

[0025] Figure 6 It is an AUC result diagram of the ERP detection of the present invention.

[0026] The realization, functional features, and advantages of the object of the present invention will be further described in conjunction with the embodiments with reference to the drawings. Detailed implementation manners

[0027] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention.

[0028] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that those of ordinary skill in the art can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0029] The present invention proposes an ERP signal detection method, as Figure 1 and 2 shown, including the following steps:

[0030] 101: Obtain RSVP electroencephalogram data, preprocess the RSVP electroencephalogram data to obtain an RSVP reference data set.

[0031] The RSVP electroencephalogram data refers to the electroencephalogram data collected when a healthy subject performs a target detection task of the rapid serial visual presentation (RSVP) paradigm.

[0032] 102: Obtain the electroencephalogram node connection relationship according to the adjacent relationship of the electrodes on the electroencephalogram cap.

[0033] 103: Construct an ERP signal detection model (as Figure 3 shown); the ERP signal detection model includes a frequency feature encoder, a spatial feature encoder, and a temporal feature encoder connected in series.

[0034] 104: Use the RSVP reference data set and the electroencephalogram node connection relationship to train and verify the ERP signal detection model in a cross-subject cross-validation manner, and determine the parameters of the ERP signal detection model.

[0035] 105: Input the preprocessed RSVP electroencephalogram data to be measured into the trained ERP signal detection model, use the frequency feature encoder, the spatial feature encoder, and the temporal feature encoder to encode and fuse the frequency, spatial, and temporal features respectively to obtain global features, and perform binary classification according to the global features to obtain a classification result, and determine whether there is an ERP signal in the RSVP electroencephalogram data to be measured.

[0036] The classification in step 105 is binary classification. If there is an ERP signal, it is Figure 3The target-present image in; without ERP signal, it is Figure 3 the target-absent image in.

[0037] In one embodiment, for step 101, preprocessing the RSVP EEG data includes:

[0038] Successively performing EEG channel electrode information assignment, useless EEG data elimination, band-pass filtering, data segment selection, and baseline correction on the RSVP EEG data to obtain the RSVP reference data set.

[0039] The acquisition process of the RSVP EEG data is as follows: The subject wears an EEG cap and watches target-present and target-absent images played at a certain frequency. The appearance of the target-present image is a low-probability event. During the process of watching the sequence of images, the RSVP EEG data of the subject is collected through the electrodes on the EEG cap.

[0040] In this embodiment, the RSVP EEG data set obtained in step 101 comes from the RSVP EEG data containing 64 subjects' EEG data on the official website of Tsinghua University, and the acquisition process is as Figure 4 shown. This RSVP EEG data uses a 64-channel acquisition device, and the electrode distribution adopts the internationally common 10-20 lead system. Assign EEG channel electrode information to the obtained RSVP EEG data, eliminate useless EEG data, perform band-pass filtering, data segment selection, and baseline correction, and then preliminarily extract feature data.

[0041] Divide the data of 64 subjects in the RSVP reference data set into test set data, validation set data, and training set data in a ratio of 4:10:50. Among them, each single sample of each subject is the EEG data, where C represents the number of channels of the EEG signal, T represents the number of sampling points, represents a real number, and the sampling frequency is 250 Hz. In this embodiment, each single sample of each subject is The total number of samples is 16000. Thus, the input data required for the ERP detection task is obtained.

[0042] In the next embodiment, for step 103, the frequency feature encoder includes a multi-scale convolutional sub-network, and the working process of the frequency feature encoder includes:

[0043] Using the multi-scale convolutional sub-network to perform three-scale transformations on the reference data of each time segment in the RSVP reference data set, and extract the frequency features under different scale transformations;

[0044] Through the splicing operation, splice the frequency features under different scale transformations, and use the 1×1 convolutional layer to fuse the frequency information extracted by different convolutional kernels of the same scale in the transformation to obtain the frequency features.

[0045] The frequency feature encoder will extract frequency features for all time segments in the RSVP benchmark dataset. Each time segment undergoes three-scale transformations through a multi-scale convolutional sub-network:

[0046]

[0047] where C represents the number of channels of the EEG signal; t′ represents the length of the feature after convolution; Q i represents the i-th subject's single sample; represents the three-scale transformations; U i ,

[0048] respectively represent the frequency features obtained from the three-scale transformations.

[0049] Each scale transformation includes convolution, batch normalization (BN), and ReLU operations. The convolutional kernels in the convolutional layer of each scale transformation are based on the current signal sampling frequency and specific ratios [0.5, 0.25, 0.125] based on the sampling frequency to extract frequency features of 2 Hz, 4 Hz, and above 8 Hz.

[0050] Finally, the features extracted from the three transformations are concatenated through a concatenation operation, and the frequency information extracted by different convolutional kernels of the same scale in the transformation is fused using a 1×1 convolutional layer to obtain the final frequency feature Y i , which is regarded as the node feature in the electroencephalogram in the input space encoder.

[0051] In this example, the sizes of the convolutional kernels used in the three transformations are [1×125], [1×62], [1×31] respectively, and the number of convolutional kernels in each branch is 20, and the output dimension of the 1×1 convolution is 1.

[0052] In another embodiment, for step 103, the spatial feature encoder includes a graph convolutional sub-network, and the working process of the spatial feature encoder includes:

[0053] Using the frequency feature and the EEG node connection relationship as Figure 5 shown, to obtain the graph representation of the EEG nodes;

[0054] According to the EEG node connection relationship, using the graph convolutional sub-network to sequentially perform feature propagation, linear transformation, and normalization on the EEG node features in the graph representation to obtain the spatial features of all EEG nodes in the graph representation;

[0055] Using the graph convolutional sub-network to aggregate the spatial features of all EEG nodes to obtain the global spatial feature.

[0056] In this embodiment, first, the frequency feature (node feature) Y is utilized i and the EEG node connection relationship G to obtain the graph representation Z of the EEG nodes i , where the nodes and the node features correspond one by one. The node connection relationship can also be expressed as G = (V, E), and V = {v1, v2,..., v C} represents all the nodes in the electroencephalogram. In the graph representation, each adjacent node relationship is expressed as E = {v p v q ∣(p, q) ∈ H}, where H represents all adjacent node pairs in the spatial domain, and p, q represent a pair of adjacent nodes. By adding self-connections to E to consider the characteristics of the nodes themselves. Thus, the adjacency matrix A is obtained through the transformation of E: Training matrix is proposed to be used to learn the connection weights of adjacent node pairs during the training process, w pq > 0 represents the connection weight between nodes v p and v q .

[0057] After completing the graph representation of the EEG nodes, according to the spatial connection relationship, the EEG node features are propagated using the graph convolutional layer in the graph convolutional subnetwork. The node features will gradually undergo feature propagation, linear transformation, and normalization. Feature propagation can be expressed as where D is the degree matrix of A, and D ii = ∑ j A ij , that is, summing all the elements in the i-th row of the adjacency matrix A can obtain the elements in the i-th row and i-th column of the degree matrix D, Z i is the graph representation of the i-th time segment, is the graph representation after feature propagation. The linear transformation matrix is used to refine the features in the nodes: where 3t′ corresponds to the length of the frequency feature, and u is the length of the hidden layer of the graph convolutional layer, and then the graph representation after refining the features is obtained Finally, batch normalization can effectively prevent feature over-smoothing and accelerate training: Z′ i is the normalized graph representation.

[0058] Finally, through the two-dimensional convolutional layer in the graph convolutional subnetwork, the spatial features in all nodes can be effectively aggregated to obtain the global spatial feature S i = Conv(Z′ i ), which contains convolution kernels of size C×1. In this embodiment, u = 100, the convolution kernel size is 62×1, and the number is 20.

[0059] In one embodiment, for step 103, the temporal feature encoder includes a recurrent attention sub-network, and the working process of the temporal feature encoder includes:

[0060] Processing the global spatial features obtained from all time segments using a two-layer long short-term memory network (LSTM), and taking the hidden state of the second layer as the output of the time features;

[0061] Inputting the time features of all time segments into the self-attention module to obtain the importance weights of the features of different time segments, and aggregating the time features of each segment according to the weights to obtain the final global features.

[0062] In this embodiment, the recurrent attention sub-network includes two layers of LSTM sub-networks and a self-attention module to learn temporal features. Each layer of LSTM includes n LSTM units corresponding to the time segments, and the hidden state of the second layer of LSTM is the output of the LSTM: where f is the number of hidden neurons.

[0063] In addition, the dropout operation in the LSTM layer is used to prevent over-smoothing.

[0064] After that, the time features h' learned from all time segments are input into the self-attention mechanism to obtain the importance weights of the features of different time segments calculated using the Softmax function where is the linear mapping obtained using the weight matrix and the offset Finally, the global spatial features are obtained through weighted summation:

[0065]

[0066] In this embodiment, the size f of each hidden layer in the two-layer LSTM sub-network is set to 256, and the size o of the hidden layer in the self-attention module is 256.

[0067] In the next embodiment, for step 103, the ERP signal detection model further includes a temporal sliding window sub-model and a classification sub-model.

[0068] The temporal sliding window sub-model is used to divide the RSVP benchmark data set into several time segments through the sliding window technique.

[0069] Q i represents each time segment obtained by segmentation; represents a real number; C represents the number of channels of the electroencephalogram signal; t represents the size of the sliding window.

[0070] The number of time segments​ where represents the floor function, and α represents the step size. In this embodiment, t = 100, α = 50, and n = 5.

[0071] The classification sub-model is used to perform binary classification based on the global features to obtain a classification result to determine whether there is an ERP signal in the to-be-tested RSVP EEG data.

[0072] The classification sub-model of this embodiment uses two fully connected layers and a Softmax function to classify the extracted global features. The number of neurons in the fully connected layers is 64 and 2 respectively.

[0073] For step 104, the loss function of the ERP signal detection model is the cross-entropy loss function,

[0074]

[0075] In the formula, and P m are the label and predicted probability of the m-th sample respectively.

[0076] The Adam optimizer is used to minimize the loss function. The optimizer can update and adjust the parameters during the training process of the model to minimize the loss function and achieve parameter optimization of the initial model. In this embodiment, the learning rate is set to 10 -3 , the batch training size is 200, Dropout is set to 0.5 to prevent overfitting, and at the same time batch normalization operation is adopted to prevent gradient disappearance and gradient explosion. The cross-validation method is used to divide the training set, validation set, and test set. First, 4 subjects are selected in sequence as the test set, then 10 subjects are randomly selected as the validation set, and the remaining 50 subjects are used as the training set. This process traverses all the subject data to obtain 16 groups of optimal models for training. To improve the training speed and reduce the impact of unbalanced data on the experimental results, the negative samples in the training set are downsampled so that the total number of positive and negative samples for each module of each subject is 1000. The sample order in the training set is random.

[0077] The ERP detection results of all subject samples in the test set in the ERP signal detection model are statistically analyzed, and the AUC value of the ERP detection is calculated as an evaluation index. The AUC value is the area under the Receiver Operating Characteristic (ROC) curve. Generally, 0.5 < AUC < 1. The larger the AUC value, the better the classification effect of the model.

[0078] In this embodiment, multiple experiments are conducted under the same experimental environment and dataset, and the experimental results are averaged to obtain the ERP detection results of the test set. The AUC values of various methods (existing HDCA, EEGNet, CRAM, MACRO, TSception, DGCNN, AMCNN-DGCN, and the MCGRAM network of the present invention) are as Figure 6 shown. Compared with deep learning methods, the performance of HDCA is the worst. The main reason is that it is difficult to extract important information in electroencephalogram signals by manually extracting features. The two GCN-based models, DGCNN and AMCNN-DGCN, only obtained AUC values of 0.9167±0.0312 and 0.8906±0.0493, respectively. This result indicates that the existing methods that only use GCN to extract and classify spatial features may not be applicable to the ERP detection task. Among all the models, the TSception, CRAM, and MACRO models have significantly better detection effects than EEGNet, DGCNN, and AMCNN-DGCN. A reasonable explanation is that these three models effectively extract global features from the frequency domain, spatial domain, and time domain. Compared with the above three methods, the method of the present invention further focuses on extracting frequency-space-time features, effectively combines multi-scale convolution, graph convolution, LSTM, and self-attention mechanism, and improves the detection results. The above results verify that the detection method proposed in the present invention (the model of the present invention is a deep learning network MCGRAM based on graph convolutional neural network and multi-scale convolutional recurrent attention model) has good performance in the subject-independent ERP detection task.

[0079] The present invention also proposes an ERP signal detection system, including:

[0080] A data acquisition module, configured to acquire RSVP electroencephalogram data, preprocess the RSVP electroencephalogram data to obtain an RSVP reference dataset; and obtain an electroencephalogram node connection relationship according to the adjacent relationship of electrodes on the electroencephalogram cap;

[0081] A model construction module, configured to construct an ERP signal detection model; the ERP signal detection model includes a series-connected frequency feature encoder, a spatial feature encoder, and a temporal feature encoder; and use the RSVP reference dataset and the electroencephalogram node connection relationship to train and verify the ERP signal detection model in a cross-subject cross-validation manner to determine the optimal parameters of the ERP signal detection model;

[0082] The detection module is used to input the preprocessed RSVP EEG data to be measured into the trained ERP signal detection model, encode and fuse the frequency, spatial and temporal features respectively by using the frequency feature encoder, spatial feature encoder and temporal feature encoder to obtain global features, perform binary classification to obtain the classification result, and judge whether there is an ERP signal in the RSVP EEG data to be measured.

[0083] The present invention also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of the above-mentioned method are implemented.

[0084] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned method are implemented.

[0085] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention. Any equivalent structural transformation made by using the content of the specification and drawings of the present invention under the inventive concept of the present invention, or direct / indirect application in other related technical fields, is included in the patent protection scope of the present invention.

Claims

1. An ERP signal detection method, characterized in that, Including the following steps: Obtain RSVP EEG data, preprocess the RSVP EEG data to obtain an RSVP reference dataset; Obtain EEG node connection relationships according to the adjacent relationships of electrodes on the EEG cap; Construct an ERP signal detection model; the ERP signal detection model includes a cascaded frequency feature encoder, a spatial feature encoder, and a temporal feature encoder; The frequency feature encoder includes a multi-scale convolutional sub-network. The working process of the frequency feature encoder includes: using the multi-scale convolutional sub-network to perform three-scale transformations on the reference data of each time segment in the RSVP reference dataset, and extracting frequency features under different scale transformations; splicing the frequency features under different scale transformations through a splicing operation, and using a 1×1 convolutional layer to fuse the frequency information extracted by different convolutional kernels of the same scale in the transformation to obtain frequency features; the spatial feature encoder includes a graph convolutional sub-network. The working process of the spatial feature encoder includes: using the frequency features and EEG node connection relationships to obtain a graph representation of EEG nodes; according to the EEG node connection relationships, using the graph convolutional sub-network to sequentially perform feature propagation, linear transformation, and normalization on the EEG node features in the graph representation to obtain the spatial features of all EEG nodes in the graph representation; using the graph convolutional sub-network to aggregate the spatial features of all EEG nodes to obtain a global spatial feature; the temporal feature encoder includes a recurrent attention sub-network. The working process of the temporal feature encoder includes: using a two-layer long short-term memory network to process the global spatial features obtained from all time segments, and taking the hidden state of the second layer as the temporal feature output; outputting the temporal features of all time segments to obtain the importance weights of the features of different time segments, and aggregating the temporal features of each segment according to the weights to obtain the final global feature; Use the RSVP reference dataset and EEG node connection relationships to train and verify the ERP signal detection model in a cross-subject cross-validation manner to determine the optimal parameters of the ERP signal detection model; Input the preprocessed RSVP EEG data to be measured into the trained ERP signal detection model, use the frequency feature encoder, the spatial feature encoder, and the temporal feature encoder to encode and fuse the frequency, spatial, and temporal features respectively to obtain a global feature, perform binary classification according to the global feature to obtain a classification result, and judge whether there is an ERP signal in the RSVP EEG data to be measured.

2. The ERP signal detection method according to claim 1, characterized in that, Preprocessing the RSVP EEG data includes: Sequentially perform operations of assigning EEG channel electrode information, removing useless EEG data, band-pass filtering, data segment selection, and baseline correction on the RSVP EEG data to obtain an RSVP reference dataset.

3. The ERP signal detection method according to claim 1, wherein The ERP signal detection model further includes a temporal sliding window sub-model and a classification sub-model; The temporal sliding window sub-model is used to divide the RSVP reference dataset into several time segments through a sliding window technique; The classification sub-model is used to perform binary classification according to the global feature to obtain a classification result, and judge whether there is an ERP signal in the RSVP EEG data to be measured.

4. The ERP signal detection method according to claim 1, wherein, The loss function of the ERP signal detection model is the cross-entropy loss function. wherein, and are respectively m the label and predicted probability of the m th sample.

5. An ERP signal detection system, characterized in that, It includes: A data acquisition module for acquiring RSVP EEG data, preprocessing the RSVP EEG data to obtain an RSVP reference data set; Obtaining the EEG node connection relationship according to the adjacent relationship of the electrodes on the EEG cap; A model construction module for constructing an ERP signal detection model; the ERP signal detection model includes a cascaded frequency feature encoder, a spatial feature encoder, and a temporal feature encoder; The frequency feature encoder includes a multi-scale convolutional sub-network. The working process of the frequency feature encoder includes: using the multi-scale convolutional sub-network to perform three-scale transformations on the reference data of each time segment in the RSVP reference data set, and extracting frequency features under different scale transformations; splicing the frequency features under different scale transformations through a splicing operation, and using a 1×1 convolutional layer to fuse the frequency information extracted by different convolutional kernels of the same scale in the transformation to obtain frequency features; the spatial feature encoder includes a graph convolutional sub-network. The working process of the spatial feature encoder includes: using the frequency features and the EEG node connection relationship to obtain the graph representation of the EEG nodes; according to the EEG node connection relationship, using the graph convolutional sub-network to sequentially perform feature propagation, linear transformation, and normalization on the EEG node features in the graph representation to obtain the spatial features of all EEG nodes in the graph representation; using the graph convolutional sub-network to aggregate the spatial features of all EEG nodes to obtain the global spatial features; the temporal feature encoder includes a recurrent attention sub-network. The working process of the temporal feature encoder includes: using a two-layer long short-term memory network to process the global spatial features obtained from all time segments, and taking the hidden state of the second layer as the temporal feature output; outputting the temporal features of all time segments to obtain the importance weights of the features of different time segments, and aggregating the temporal features of each segment according to the weights to obtain the final global feature; using the RSVP reference data set and the EEG node connection relationship, training and validating the ERP signal detection model in a cross-subject cross-validation manner to determine the optimal parameters of the ERP signal detection model; A detection module for inputting the preprocessed RSVP EEG data to be measured into the trained ERP signal detection model, encoding and fusing the frequency, spatial, and temporal features respectively by using the frequency feature encoder, the spatial feature encoder, and the temporal feature encoder to obtain the global feature, performing binary classification according to the global feature to obtain the classification result, and determining whether there is an ERP signal in the RSVP EEG data to be measured.

6. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method described in any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the method described in any one of claims 1 to 4.

Images