A cross-subject RSVP electroencephalogram signal classification method based on transfer learning

Abstract

This paper presents a cross-subject RSVP EEG signal classification method based on transfer learning, belonging to the field of computer software. It addresses the issue of the RSVP-BCI system requiring significant time for calibration when changing subjects. First, a source-domain subject selection algorithm (MSS) combining outcome-based and distance-based methods is used to screen subjects from multiple source domains, reducing training time and mitigating or eliminating negative transfer to some extent. Then, an ADCN employs a windowed deep convolutional model to analyze the RSVP EEG signals in the spatiotemporal domain. An anti-adversarial network (AAN) is added to distinguish between the source domain and the current subject, enabling the network to extract common features from both domains before final classification. Compared to the ADCN algorithm without MSS, the addition of MSS reduces training time while improving classification accuracy. MSS-ADCN achieves acceptable accuracy with a limited number of calibration trials provided by the subject and outperforms other transfer learning algorithms.

Description

Technical Field

[0001] This invention discloses a classification method for cross-subject RSVP EEG signals based on source domain selection and deep transfer model, which can be used for cross-subject RSVP signal recognition and belongs to the field of computer software. Background Technology

[0002] The Rapid Serial Visual Presentation (RSVP) paradigm, first proposed by Mary C. Potter, is defined as the process of presenting multiple images sequentially at a high and fixed frequency within the same spatial location. A Brain-Computer Interface (BCI) is a communication and control system established between a human and a machine, where the human's brainwave activity controls the machine to complete tasks. The image stream presented by the RSVP paradigm consists of frequently occurring non-target images and infrequently occurring target images. When the subject observes the target image, the P300 component is induced, and through decoding, the corresponding target image can be identified. Currently, RSVP-BCI plays an important role in BCI enhancement, police investigation, medicine, cognitive science, and psychology. In practical applications, due to the low signal-to-noise ratio caused by the high dimensionality of EEG signals, the high instability of the signals, and the different brainwave patterns of each subject, when a new subject is introduced, a lengthy calibration process is required to provide calibration data so that the model can be trained to adapt to the current subject. This lengthy calibration process reduces the efficiency of RSVP-BCI and the user experience.

[0003] To address the aforementioned issues, transfer learning methods are frequently used in research on cross-subject classification. Generally, these methods fall into three different categories: instance-based, feature-based, and model-based.

[0004] In instance-based transfer learning, the basic idea is to measure the similarity between source and target domain data, i.e., the number of trials by subjects in the source domain and the number of trials by subjects in the target domain. Trials from the source domain that are more similar to the target trials are given higher weights, contributing a larger loss value to the loss function, thereby transferring data from the source domain. For example, Yan Li et al. proposed Bagged-IWLDA based on LDA and Bagging methods. IWLDA addresses the problem of adapting to covariance shifts between different domains. It is a derivative of LDA based on the concept of sample weighting, where the weight is defined as the ratio of the density of a sample to the density of the target sample. However, estimating the sample distribution density is difficult. The paper points out that KLIEP or uLSIF can be used to estimate the weights. Using only IWLDA may result in a classifier with high variance, leading to unstable classification performance. To address this issue, the Bagging concept is introduced, which improves classification performance by averaging the results of multiple classifiers as the final result. On the BCI2000 dataset, cross-day testing showed that BIWLDA accuracy improved from 0.9336 to 0.965 compared to IWLDA, and from 0.8115 to 0.965 compared to BLDA. Similarly, Yang Liu et al. and Jiayun Hou et al. conducted cross-subject transfer experiments on the P300 Speller dataset. They used TrAdaBoost on SVM methods, which are represented by traditional machine learning. During the loop, they mainly performed two tasks: first, assigning weights to different samples, with the weights reflecting the prediction accuracy of the current subject dataset on the test set; second, balancing the N obtained SVM classifiers, also using accuracy as the weight, to obtain the final classifier.

[0005] In feature-based transfer learning, the basic idea is to create a feature space on the source and target domain trials, achieving unification between the source and target domain trials within this feature space. Farah Abid et al. proposed the mSDAs method based on the Autoencoder concept, implemented through a Denoising Autoencoder. Compared to the traditional KMM method, this method has lower data requirements, not requiring every sample to be valid and labeled, and shows approximately a 5% accuracy improvement over KMM in cross-day experiments. Furthermore, due to the importance of the covariance matrix in EEG signals, some researchers have proposed using the covariance matrix to achieve feature unification. For example, Hyohyeong Kang et al. generated the final covariance matrix by linearly combining the covariance matrices of the target and source domains. They provided two different methods for constructing this linear combination: the first method favors subjects with more trials, while the second method focuses on source domain data similar to the target domain (measured by KL divergence). Similarly, in cross-subject experiments, their results outperformed the CSP and SVM methods. Dieter Devlaminck et al. proposed constructing a shared-space filter by reconstructing the CSP optimization function. The main idea is that this shared-space filter consists of two parts: a global filter (trained from all subjects) and a specific filter (trained from the current subject). The final optimization function is determined by minimizing the optimization function generated by the shared filter. Unlike the above methods, another type of feature-based transfer learning focuses on alignment. For example, the RA method is based on Riemannian space and uses specific reference data (different paradigms have different feature reference data) to achieve alignment in Riemannian space. Similarly, the EA method is based on Euclidean space and uses the data itself as a reference to achieve whitening of the aligned source and target domain data. Other similar methods include LA and PA.

[0006] In model-based transfer learning, the basic idea is to train the model on the source domain trials and then directly modify it for the target trials. For example, the method proposed by Wenting Tu et al. in their cross-subject transfer learning approach first trains a classifier for each subject, then constructs two sub-classifiers from these classifiers using different criteria: a robust filter and an adaptive filter. Finally, these two classifiers are integrated to obtain the final classifier. Another common approach is to use fine-tuning on Convolutional Neural Networks (CNNs) to achieve transfer learning. In CNN models, lower convolutional layers typically extract general features, while higher layers gradually specialize those features. Therefore, in this method, the parameters of the early feature extraction layers (CNNs) are trained using other subject datasets, while the current dataset only updates the parameters of the Dense layer and subsequent layers.

[0007] By discussing and analyzing the advantages and disadvantages of existing research methods, this invention provides new inspiration and research ideas, proposing a method based on source domain selection and deep transfer learning, abbreviated as "MSS-ADCN" (Mixed Source Selection-Adversarial Deep Convolutional Net). First, the MSS method, which combines outcome-based and similarity-based methods, is used to screen subjects from multiple source domains, reducing training time and, to some extent, reducing or eliminating negative transfer phenomena. Then, ADCN employs a windowed deep convolutional model to analyze RSVP EEG signals in the spatiotemporal domain, incorporating an adversarial network to distinguish between the source domain and the current subject, enabling the network to extract common features from both for final classification. Compared to ADCN without MSS, incorporating MSS reduces training time while improving classification accuracy. MSS-ADCN achieves acceptable accuracy with a limited number of calibration trials provided by the subject and outperforms other transfer learning methods. Summary of the Invention

[0008] This invention proposes a classification method for cross-subject RSVP EEG signals based on source domain selection and deep transfer models, which can effectively improve classification performance in cross-subject RSVP EEG signal scenarios. To address the problem of numerous source domain subjects and high uncertainty, the MSS (MixedSourceSelection) method is proposed. This method integrates result-based and similarity-based source domain selection methods to screen multiple source domain subjects, reducing training time and minimizing or eliminating negative transfer phenomena. Subsequently, an ADCN (Adversarial Deep Convolutional Net) model is designed. This model uses a windowed deep convolutional model to sequentially decode and analyze the temporal and spatial information of RSVP and extract latent features. To eliminate the gap between source domain trials and the current trial, an adversarial network is introduced. This network distinguishes whether the input trial originates from the source domain or the target domain. The loss function of this network is passed through a GRL (gradient reverse layer), which ensures that the network operates normally during forward propagation but is negative during backward propagation, effectively maximizing the loss function. This allows the network to extract common features between the target and source domain trials. Compared to ADCN methods without MSS, adding MSS reduces training time while improving classification accuracy. MSS-ADCN achieves acceptable accuracy with a small number of calibration trials provided by the subjects and outperforms other transfer learning methods.

[0009] After research, discussion, and repeated practice, the final solution determined by this method is as follows:

[0010] First, the original RSVP EEG dataset is preprocessed, including trial segmentation, bandpass filtering, and baseband removal. Then, each subject is used as the current subject in turn, and is divided into a training set and a test set. The remaining subjects are used as the source domain subjects, forming the training set. The training sets of the source domain subjects and the current subject are input into the MSS method to derive the filtered subject-derived data. The filtered subject-derived data and the current subject's training data are then input into the ADCN model for training. Finally, the current subject's test data is input into the trained ADCN to obtain the classification results. The classification results are evaluated and analyzed to verify the effectiveness of the method.

[0011] The specific steps of the technical solution of this invention are as follows:

[0012] Step 1. Data Preprocessing: The RSVP EEG signal is divided into trials, with each trial defined as the period from the 0th to the 1st second of the trigger occurrence. A bandpass filter is used to perform bandpass filtering on the divided RSVP EEG signal trials. Baseband removal is performed on the filtered signal using the data within each trial. Channel-based standardization is then applied to the trial data. One subject is selected alternately as the current subject, and a training set and a test set are created. The remaining subjects are used as the source domain subjects, forming the training set.

[0013] Step 2. Build the MSS model. Input the source domain trials into the MSS model. Through model calculation, obtain the selected K source domain subjects as the final source domain training set.

[0014] Step 3. Build the ADCN model by integrating the current subject's training set and the source domain training set, and then inputting them into the ADCN model for training.

[0015] Step 4. Input the current subject's test set into the trained ADCN model to obtain the classification results, compare them with the original labels, and evaluate the model's classification performance.

[0016] The present invention has the following advantages:

[0017] 1. An additional source domain subject screening method, MSS, is added. This method combines two different perspectives to screen source domain subjects, reducing the training time of ADCN and reducing or eliminating negative transfer phenomenon.

[0018] 2. The windowed CNN model ADCN was used to analyze RSVP EEG signals, with an emphasis on the time domain. An anti-adversarial network was added to the model, enabling the network to extract common features from the source and target domains. Attached Figure Description

[0019] Figure 1 This is the overall flowchart of the method of the present invention.

[0020] Figure 2 This is a schematic diagram of MSS.

[0021] Figure 3 This is a diagram of the ADCN network structure.

[0022] Figure 4 This is a time-based diagram of the RSVP dataset. Detailed Implementation

[0023] This invention addresses the drawback of RSVP brain-computer interfaces requiring lengthy calibration processes when switching subjects by proposing a cross-subject RSVP EEG signal classification method based on transfer learning. This method employs a combined source domain subject screening method (MSS) based on outcome and distance, reducing training time and eliminating or minimizing negative transfer. Subsequently, an ADCN model is used to train the selected source and target domain subjects on trials. This model uses a windowed RSVP structure to analyze RSVP signals and incorporates a GRL (Gross Adversarial Network) to narrow the gap between the source and target domains, enabling target domain subjects to achieve good accuracy with only a limited number of trials.

[0024] Figure 1 The overall flowchart of the method of this invention can be broken down into the following steps:

[0025] Step 1. Read the data from the source domain participants and the current participant.

[0026] Step 2. Data Preprocessing

[0027] Step 3. MSS source domain subject screening

[0028] Step 4. Train the ADCN model

[0029] Step 5. Input the test set of the target subjects into the trained ADCN model, obtain the test results, and analyze the results.

[0030] The specific details of each step are explained below:

[0031] Step 1:

[0032] (1) Select one subject as the target subject, and the rest as the source subject.

[0033] (2) Randomly split the current subject's trial into a training set and a test set, ensuring that the proportion of each category is consistent in each set.

[0034] Step 2:

[0035] (1) Read the data, retain the EEG data, remove the electrooculogram channel, and downsample to 250Hz.

[0036] (2) Perform bandpass filtering from 0 to 30.

[0037] (3) Divide the time into epochs. The duration from the start of the stimulus to the first second is cut off as one epoch, and this data is used as the standard for baseline removal.

[0038] Step 3:

[0039] First, distance-based calculations are performed for each trial of the source domain and the target subject, as follows:

[0040] A common response feature matrix should exist for a specific category of photographs across different trials: A∈R C×T Where R represents the set of real numbers, C is the number of EEG data channels, and T is the number of EEG data time points; and for different trials, there should be a specific expression belonging to the current trial, which is generated by the comprehensive transformation of space and time on the common feature matrix, so the channel space specific transformation matrix is ​​defined as: D i ∈R C×C (i = 1, 2…N) and time-specific transformation matrix: U i ∈R T×T (i = 1, 2, ..., N), where N is the total number of trials for the current subject. Thus, the EEG signal X of one trial... i ∈R C×T It can be decomposed into:

[0041] X i =D i T AU i +Noise#(1)

[0042] Wherein, "Noise" refers to the noise component;

[0043] To minimize the noise in each trial, an optimization function can be established:

[0044]

[0045] in, To minimize the noise across all trials for the subject, so that It is closer to the original EEG signal.

[0046] Note that the above optimization formula cannot be solved in one step; here we draw on the self-taught approach.

[34] The solution method uses iterative gradient descent, because D i And U i Only with X i It is related, meaning it can be updated in one trial, but A and X i The solution process is related to (i = 1…N) and can only be updated after one iteration. The solution process is as follows:

[0047] In the iteration process, t represents the number of iterations. This represents the t-th iteration of the D matrix in the i-th trial; A represents the t-th iteration of the U matrix in the i-th trial; t Let M represent the t-th iteration of matrix A; M is the maximum number of iterations. By calculating the gradient, we can obtain:

[0048]

[0049] Therefore The updated formula is:

[0050]

[0051] Where α represents the learning rate, which is set to 0.01.

[0052] Similarly, for By calculating the gradient, we can obtain:

[0053]

[0054] Therefore U t+1 i The updated formula is:

[0055]

[0056] Specific expression for all trials in one round as well as After solving, the common characteristic matrix A can be obtained. t To solve this problem, in the above optimization formula, for A... t By calculating the gradient, we can obtain:

[0057]

[0058] Therefore, after one round of iterations (all trials for the current subject), A can be... t Update:

[0059] A t+1 =A t -αA t #(8)

[0060] Using the above method, the common response feature matrix A of all subjects can be obtained. The subjects are divided into source domain subjects and target domain subjects, where... It is the common response feature matrix of subjects from all source domains, while A T The shared response feature matrix of the target subjects is used. The L2 normal form is performed on the A matrix for each source domain subject and target subject to obtain... That is, source domain filtering score based on distance:

[0061]

[0062] Subsequently, source domain participant selection based on the results is calculated. The ADCN model is trained using the trials of each source domain participant, and then the trials of the target participant are fed into each ADCN model to obtain the Average ACC classification result, i.e., S. i .

[0063] The two types of results are linearly weighted to obtain the final score f for each source domain participant. i These scores are sorted from largest to smallest, and K subjects are selected as the final source domain subjects to participate in the training set of ADCN. Figure 2 The above method and process are demonstrated.

[0064]

[0065] Step 4:

[0066] To address the challenges of feature extraction and classification caused by the large difference between source and target domain trials and the low signal-to-noise ratio of EEG signals, a windowed RSVP deep convolutional model is proposed. In this model, GRL introduces an inverse adversarial network to narrow the gap between the source and target domains. Figure 3 The network structure is shown, mainly divided into four parts: Block 1, Block 2, Block 3, and Block 4. Each part is described in detail below:

[0067] (1)Block1

[0068] Block 1 primarily performs time-domain analysis on the RSVP EEG signal. First, the raw RSVP EEG signal is windowed with a window size of (59, 125) and a stride of 25, resulting in a RSVP EEG data structure of (59, 125, 6). Then, eight 2D convolutional layers of size (1, 60) are used to convolve the windowed RSVP EEG signal to extract the frequency band above 4Hz. Batch Normalization is then performed, followed by processing using the ELU activation function. To prevent overfitting, a Dropout layer with a parameter of 0.5 is introduced. Next, a single 2D convolutional layer of size (1, 31) is used to obtain a (36, 48, 59) convolutional layer, which is then processed again using Batch Normalization and the ELU activation function.

[0069] (2) Block2

[0070] Block2 primarily performs spatial dimension analysis. A deep convolutional layer of size (59,1) is used for channel analysis, followed by BatchNormalization and ELU activation. The result is then subjected to 2D average pooling of size (1,8) and stride (1,4) to reduce dimensionality, resulting in a data structure of size (1,8,48). This is further modified to (1,48,8) using a reshape layer. To prevent overfitting in the spatial dimension, Dropout with a parameter of 0.5 is applied.

[0071] (3) Block3

[0072] After extracting information in the temporal and spatial domains, 16 depthwise separable convolutional layers of size (1,16) are used to integrate and analyze the information in the temporal and spatial domains. Then, average pooling of size (1,13) and stride (1,3) is used to reduce dimensionality, and a dropout layer with parameter 0.5 is used to reduce the risk of overfitting. Finally, a convolutional result of (1,8,16) is obtained.

[0073] (4) Block4

[0074] Block 4 is mainly divided into two parts: a classification network dominated by the original classification labels and an adversarial network introduced by GRL. First, the convolution result of (1,8,16) is flattened to obtain a 128-bit feature vector, followed by a Dense layer to obtain a 64-bit feature vector. For the classification network, a Dense layer is used to obtain a classification result with a feature length of 3, which is then passed through softmax to obtain the final classification probability. This classification result uses the original classification labels (0,1,2) as the comparison result for the loss function. For the adversarial network, a Dense layer is used to obtain a classification result with a feature length of 2, which is then passed through softmax to obtain the final classification probability. This classification result uses the domain classification labels (0,1) as the comparison result for the loss function. Because this network has passed through GRL layers, the loss is negatively charged during backpropagation.

[0075] Step 5:

[0076] The test set of the target subjects is input into the trained ADCN model to obtain the corresponding classification results. The classification accuracy is then analyzed by comparing the results with the original class labels.

[0077] The dataset and experimental results used in the method of this invention are described below:

[0078] 1. Dataset

[0079] We used the RSVP dataset from the 2021 and 2022 World Robot Competition ERP track. This dataset contains three classes of images, corresponding to people (“target-1”), vehicles (“target-2”), and backgrounds with no people or vehicles (“nontarget”). All images presented to the participants were from street view images.

[0080] Data was collected from each participant in blocks. In this experiment, each participant collected data from multiple blocks. Before each block began, a prompt appeared in the center of the screen. The image sequence was presented in trials. At the start of each trial, a crosshair would appear on the screen prompting the participant to focus on the center of the screen. Each trial contained 50 images, with the type and number of targets varying (maximum 5 target images). Each image was presented in the center of the screen at a rate of 10 images per second. Each block contained 10 trials.

[0081] Experimental data were acquired using a Borrecom 64-channel EEG acquisition device. Lead 65 was used for trigger information. The raw sampling rate was 1000Hz, and no other processing was performed.

[0082] 2. Experimental Results and Discussion

[0083] To verify the effectiveness and universality of the method of the present invention, MSS and ADCN were verified on the above datasets respectively.

[0084] (1) MSS (MixedSourceSelection) verification results

[0085] To verify the effectiveness of MSS in screening source domain participants and the value of K, we statistically analyzed the results for different K values. First, we randomly selected one participant as the current participant, and the remaining participants as source domain participants. Then, we varied K from 1 to 11, meaning the number of source domain participants selected increased from 1 to 11. We then repeated this process for each participant and averaged all the results, obtaining the following:

[0086]

[0087]

[0088] The results above show that from 1 to 7, as the K value increases, the Average ACC also gradually increases, indicating that there are significant differences among different subjects, making it difficult to transfer knowledge from a single subject to new subjects. However, with an increase in the number of trials, more transferability can be provided. But when K reaches 7, the Average ACC begins to decrease, indicating that negative transfer begins to occur when K is too large. This figure demonstrates the necessity and effectiveness of MSS in selecting source domain subjects, not only reducing training time but also reducing or even eliminating negative transfer.

[0089] (2) MSS-ADCN Validation Results

[0090] To verify the overall performance of the MSS-ADCN model in RSVP classification across subjects, we reduced the number of training trials for the current subject to varying degrees (in units of one block). We observed the cross-subject performance after using MSS-ADCN:

[0091] Block EEGNet MSS-ADCN 3 0.4122 0.4912 6 0.4523 0.5998 9 0.5211 0.7271 12 0.5631 0.7731 21 (No cross-subject transfer) 0.6521 0.7192

[0092] The original dataset had a training and testing ratio of 21 blocks and 9 blocks. We used the MSS-ADCN method to reduce the proportion of the training set. As shown in the table above, the classification performance exceeded that of the original training set ratio (21 blocks) when using 9 blocks, and it also exceeded that when the training set reached 12 blocks. All results were higher than the classification performance of EEGNet. These results demonstrate that using MSS-ADCN allows for a shorter calibration process to achieve acceptable classification accuracy after changing new subjects, significantly reducing calibration time.

Claims

1. A classification method for cross-subject RSVP EEG signals based on transfer learning, characterized in that, Includes the following steps: Step 1. Read data from the source domain participants and the current participant. Step 2. Data Preprocessing: Divide the RSVP EEG signal into trials, with each trial consisting of the period from the 0th to the 1st second of the trigger occurrence; apply a bandpass filter to the divided RSVP EEG signal trials; use the data within each trial to remove the baseband from the filtered signal; perform channel-based standardization on the trial data; select one subject in turn as the current subject, dividing the data into a training set and a test set, while the remaining subjects are used as the source domain subjects and the training set. Step 3. MSS Source Domain Subject Selection: Input the source domain trials into the MSS model, and through model calculation, obtain the selected K source domain subjects as the final source domain training set. Step 4. Train the ADCN model: Build the ADCN model by integrating the current subject's training set and the source domain training set, and then inputting them into the ADCN model for training. Step 5. Input the current subject's test set into the trained ADCN model to obtain the classification result; Step 1 is as follows: (1) Select one subject as the current subject, and the rest of the subjects as the source domain subjects; (2) Randomly split the current subject's trial into a training set and a test set to ensure that the proportion of categories in each set is consistent; Step 2 is as follows: (1) Read the data, retain the EEG data, remove the electrooculogram channel, and downsample to 250Hz; (2) Perform bandpass filtering from 0 to 30. (3) Divide the epochs. The duration from the start of the stimulus to the first second is cut off as one epoch, and this data is used as the standard for baseline removal. Step 3 specifically involves: First, distance-based calculations are performed for each source domain and the current subject's trial, as follows: A common response feature matrix should exist for a specific category of photographs across different trials: Where R represents the space of the real numbers, For the number of EEG data channels, This represents the number of time points in the EEG data; and for different trials, there should be a specific expression belonging to the current trial. This specific expression is generated by the comprehensive transformation of space and time on the common feature matrix. Therefore, the channel space specific transformation matrix is ​​defined as follows: and time-specific transformation matrix: ,in This represents the total number of trials for the current subject; thus, the EEG signal of one trial... It can be decomposed into: ； in, It consists of noise; To minimize the noise in each trial, an optimization function can be established: ； in, To minimize the noise across all trials for the subject, so that It is closer to the original EEG signal; Note that the above optimization formula cannot be solved in one step, so iterative gradient descent is used for the solution. as well as Only with It is relevant, meaning it can be updated in a single trial, but and This is relevant and can only be updated after one iteration. The solution process is as follows: During the iteration process Indicates the number of iterations. Indicates the first One trial The first of the matrix The next iteration; Indicates the first One trial The first of the matrix The next iteration; express The first of the matrix The next iteration; The maximum number of iterations; for By calculating the gradient, we can obtain: ； Therefore The updated formula is: ； ； in, This represents the learning rate, set to 0.

01. Similarly, for By calculating the gradient, we can obtain: ； Therefore The updated formula is: ； ； Specific expression for all trials in one round as well as After solving, the common characteristic matrix is... To solve this problem, in the above optimization formula, for By calculating the gradient, we can obtain: ； Therefore, after one iteration, for Update: ； Using the above method, the common response feature matrix of each subject can be obtained. The subjects were divided into source domain subjects and current domain subjects, among which It is the common response feature matrix of subjects from all source domains, and This serves as the shared response feature matrix for the current subject; for each source domain subject and the current subject... To obtain the matrix using L2 normal form That is, the source domain selection score based on distance: ； Subsequently, source domain participant selection based on the results is calculated; the ADCN model is trained using the trials of each source domain participant, and then the current participant's trials are fed into each ADCN model to obtain the Average ACC classification result, i.e. ; The two types of results are linearly weighted to obtain the final score for each source domain participant. These ratings are sorted from largest to smallest, and K subjects are selected as the final source domain subjects to be selected, that is, the training set to participate in ADCN. Step 4 specifically involves: The network is divided into four parts: Block 1, Block 2, Block 3, and Block 4; each part is described in detail below: (1) Block1 Block 1 performs temporal analysis on RSVP EEG signals. First, the original RSVP EEG signal is windowed with a window size of (59, 125) and a stride of 25, resulting in a RSVP EEG data structure of (59, 125, 6). Then, eight 2D convolutional layers of size (1, 60) are used to convolve the windowed RSVP EEG signal to extract frequency bands above 4Hz. Batch Normalization is then performed, followed by processing using the ELU activation function. To prevent overfitting, a Dropout layer with a parameter of 0.5 is introduced. Finally, a 2D convolutional layer of size (1, 31) is used to obtain a convolutional layer of size (36, 48, 59), which is then processed using Batch Normalization and the ELU activation function. (2) Block2 Block2 performs spatial dimension analysis; a deep convolutional layer of size (59,1) is used for channel analysis, followed by BatchNormalization and ELU activation function processing; then, the result is processed by two-dimensional average pooling of size (1,8) and stride (1,4) to reduce the dimension to obtain a data structure of size (1,8,48), which is then changed to (1,48,8) through a reshape layer; to prevent overfitting in the spatial dimension, Dropout with parameter 0.5 is added for processing. (3) Block 3 After extracting information in the temporal and spatial domains, 16 depthwise separable convolutional layers of size (1,16) are used to integrate and analyze the information in the temporal and spatial domains. Then, average pooling of size (1,13) and stride (1,3) is used to reduce the dimensionality, and a dropout layer with parameter 0.5 is used to reduce the risk of overfitting. Finally, a convolutional result of (1,8,16) is obtained. (4) Block 4 Block 4 consists of two parts: a classification network dominated by the original classification labels and an adversarial network introduced by GRL. First, the convolution result of (1, 8, 16) is flattened to obtain a 128-bit feature vector, followed by a Dense operation to obtain a 64-bit feature vector. For the classification network, a Dense layer is used to obtain a classification result with a feature length of 3, which is then passed through softmax to obtain the final classification probability. This classification result uses the original classification labels (0, 1, 2) as the comparison result of the loss function. For the adversarial network, a Dense layer is used to obtain a classification result with a feature length of 2, which is then passed through softmax to obtain the final classification probability. This classification result uses the domain classification labels (0, 1) as the comparison result of the loss function, and the loss is negatively added during backpropagation. Step 5 specifically involves: The test set of the current subject is input into the trained ADCN model to obtain the corresponding classification results.

Images