Multi-source domain adversarial electroencephalogram signal classification method and system based on multi-task learning

Abstract

The invention discloses a multi-source domain adversarial electroencephalogram signal classification method and system based on multi-task learning. The method comprises the steps that electroencephalogram signal feature samples of a target domain O and a plurality of source domains S are obtained respectively; performing time scrambling on the electroencephalogram signal characteristic sample of each source domain S to form a plurality of electroencephalogram signal characteristic samples of a time sub-domain P; mixing the electroencephalogram signal feature sample of each source domain S with the electroencephalogram signal feature sample of the target domain O to form electroencephalogram signal feature samples of a plurality of mixed sub-domains Q; inputting the electroencephalogram signal feature samples of the source domain S, the time sub-domain P and the mixed sub-domain Q into a domain adversarial neural network for training to form a branch prediction result; and performing voting prediction on the branch prediction result to determine a final prediction result of the target domain O. According to the method, the learning ability of the multi-source domain adversarial model is enhanced by introducing multiple adversarial tasks irrelevant to the tasks, so that the robustness and generalization of the features are improved, and the accuracy of electroencephalogram signal classification prediction is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram signal processing, and in particular relates to a multi-source domain adversarial electroencephalogram signal classification method and system based on multi-task learning. Background Art

[0002] In recent years, significant progress has been made in feature extraction and classification based on electroencephalogram (EEG). However, there are significant individual differences in EEG signals, and the distribution of EEG data of different subjects is inconsistent. Therefore, cross-individual emotion recognition still faces challenges in the field of emotion recognition. In order to develop cross-individual emotion recognition models with strong generalization performance, domain adversarial algorithms have become the mainstream research direction in recent years. In this method, the EEG data of different subjects are regarded as different domains, the training subject data is regarded as the source domain, and the test subject data is the target domain. It conducts adversarial learning by designing adversarial discriminators and task classifiers, with the goal of reducing domain-related features while retaining task-related features, thereby improving the generalization ability of the model. However, although the existing domain adversarial algorithms can reduce subject-related features, the adversarial tasks designed by the model are relatively single, and the influence of interference from task-irrelevant features on subject differences is not considered, which limits the generalization ability and accuracy of the model in cross-subject tasks. Therefore, overcoming these challenges and further improving the performance of EEG-based feature extraction and classification have become key technical issues that need to be solved urgently. Summary of the invention

[0003] Technical problem to be solved by the present invention: In view of the above-mentioned problems in the prior art, a multi-source domain adversarial EEG signal classification method and system based on multi-task learning are provided. The present invention aims to enhance the learning ability of the multi-source domain adversarial model by introducing a variety of task-independent adversarial tasks to improve the robustness and generalization of features and improve the accuracy of EEG signal classification prediction.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is: A multi-source domain adversarial EEG signal classification method based on multi-task learning includes the following steps: obtaining the target domain O and multiple source domains S EEG signal feature samples; each source domain S The EEG signal feature samples are respectively scrambled to form time subdomains P Multiple EEG signal feature samples; each source domain S The EEG signal feature samples are respectively O The EEG signal feature samples are mixed to form multiple mixed sub-domains Q EEG signal feature samples; the source domain S , Time subdomain P , mixed subdomainQ The EEG signal feature samples are input into the domain adversarial neural network for training to form branch prediction results; the branch prediction results are voted to determine the target domain O The final prediction result.

[0005] Optionally, each source domain S Multiple time subdomains are formed by time scrambling P When including the source domain S The EEG signal feature samples in the image are divided into the left hemisphere and the right hemisphere. For each pair of two EEG signal feature samples at time t and time t+1 that are adjacent in time, the left hemisphere part of the EEG signal feature sample at time t and the right hemisphere part of the EEG signal feature sample at time t+1 are spliced ​​and combined to obtain the time subdomain P The EEG signal feature sample at time t is obtained by splicing the left hemisphere part of the EEG signal feature sample at time t+1 and the right hemisphere part of the EEG signal feature sample at time t to obtain the time subdomain P The EEG signal feature sample at time t+1 finally forms the time subdomain P Multiple EEG signal feature samples.

[0006] Optionally, each source domain S The EEG signal feature samples are respectively O The EEG signal feature samples are mixed to form multiple mixed sub-domains Q The EEG signal feature samples refer to the EEG signal feature samples in different domains under the same stimulus, which are mixed in equal proportions to participate in the training and source domains. S and target domain O Forming multiple hybrid subdomains Q EEG signal feature samples.

[0007] Optionally, the domain adversarial neural network includes: a sample pairing module for pairing the input source domain S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are respectively O The EEG signal feature samples are combined and paired to form various types of EEG signal feature sample pairs; the feature representation module is used to extract features from the EEG signal feature samples from the source domain or the target domain; the adversarial discriminator is used to determine the domain type of the input; the task classifier is used to determine the task type of the input.

[0008] Optionally, the source domain to be input S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are respectively OWhen the EEG signal feature samples are combined and paired, the various types of EEG signal feature sample pairs formed include the following six types of EEG signal feature sample pairs: (1) Same label, same source domain: both samples belong to the same label and both come from the source domain; (2) Different labels, same source domain: both samples belong to different labels and both come from the source domain; (3) Same label, different domains: both samples belong to the same label, but come from the source domain and the target domain respectively; (4) Different labels, different domains: two samples belong to different labels and come from the source domain and the target domain respectively; (5) Same label, same target domain: both samples belong to the same label and both come from the target domain; (6) Different labels, same target domain: two samples belong to different labels and both come from the target domain.

[0009] Optionally, the feature representation module is a multilayer perceptron MLP with shared parameters, the multilayer perceptron MLP comprises two fully connected hidden layers and one output layer; the adversarial discriminator consists of two fully connected layers and one output layer; the task classifier consists of two fully connected layers and one output layer.

[0010] Optionally, the source domain S , Time subdomain P , mixed subdomain Q Before the EEG signal feature samples are input into the domain adversarial neural network for training to form branch prediction results, the step of training the domain adversarial neural network is also included, and the loss function used for training and optimizing the network parameters of the branch network is the classification loss and domain loss The sum of the two, classification loss The calculation function expression is: , In the above formula, is the number of samples, is the task label, represents the task classifier, and Indicates task label The feature representation obtained by extracting the two EEG signal feature samples in the corresponding EEG signal feature sample pair by the feature representation module; domain loss The calculation function expression is: , In the above formula, represents the mathematical expectation, is the EEG signal feature sample pair The type tag, represents the adversarial discriminator, For splicing operation, and is the EEG signal feature sample pair The two EEG signal feature samples in are extracted by the feature representation module to obtain the feature representation.

[0011] In addition, the present invention also provides a multi-source domain adversarial EEG signal classification method based on multi-task learning, comprising a microprocessor and a memory connected to each other, wherein the microprocessor is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning.

[0012] In addition, the present invention also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning through a processor.

[0013] In addition, the present invention also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning through a processor.

[0014] Compared with the prior art, the present invention mainly has the following advantages: 1. The present invention enhances the generalization ability of the model between different individuals by introducing multiple task-independent adversarial tasks. This multi-task adversarial training method helps the model learn more generalized feature representations.

[0015] 2. The present invention designs two specific adversarial tasks, such as domain mixing and temporal scrambling, to expand the source domain S into a temporal subdomain P and a mixed subdomain Q. The expanded subdomains can be successfully introduced into domain adversarial learning, effectively retaining the features related to the task while eliminating the features irrelevant to the task, thereby achieving better performance in the cross-individual EEG recognition task. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 Schematic diagram of the basic flow of the method of the embodiment of the present invention.

[0017] Figure 2 Schematic diagram of generating the time subdomain P in an embodiment of the present invention, where (a) is the source domain S (a) is a set of EEG signal feature sample pairs in the time subdomain P, and (b) is a corresponding set of EEG signal feature sample pairs in the time subdomain P.

[0018] Figure 3 This is a schematic diagram of constructing a domain adversarial network in an embodiment of the present invention.

[0019] Figure 4 Schematic diagram of the sample pairing module structure in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] like Figure 1 As shown, the multi-source domain adversarial EEG signal classification method based on multi-task learning in this embodiment includes the following steps: obtaining the target domain O and multiple source domains S EEG signal feature samples; each source domain S The EEG signal feature samples are respectively scrambled to form time subdomains P Multiple EEG signal feature samples; each source domain S The EEG signal feature samples are respectively O The EEG signal feature samples are mixed to form multiple mixed sub-domains Q EEG signal feature samples; the source domain S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are input into the domain adversarial neural network for training to form branch prediction results; the branch prediction results are voted to determine the target domain O The final prediction result.

[0022] When the subject performs a specific task, the cerebral cortex is stimulated by the outside world and generates evoked electrical activity associated with it. The multi-source domain adversarial EEG signal feature analysis method based on multi-task learning in this embodiment selects EEG signals, an objective physical signal, to realize task classification, effectively improving the accuracy and generalization ability of task classification. The acquisition of EEG signal features in this step includes: S201: denoising the multi-channel EEG signals obtained in a single induced experiment; S202: dividing the denoised EEG signals according to fixed time windows; S203: fast Fourier transforming the EEG signals in each time window, extracting the frequency domain features of each channel, and obtaining the differential entropy features of the five frequency bands of δ, θ, α, β and γ. It should be noted that the calculation method of the differential entropy feature is an existing well-known method, and the specific method can be found in the literature: Duan R, Zhu J, Lu B. Differential entropy feature for EEG-based emotion classification. 6th International IEEE / EMBS Conference on Neural Engineering (NER), San Diego, CA, USA. 2013.6695876.

[0023] The noise reduction in step S201 can adopt a feasible noise reduction method as needed. For example, as an optional implementation, the noise reduction in step S201 of this embodiment includes: S301, downsampling the original EEG signal to reduce the amount of data and improve the efficiency of subsequent processing; S302, baseline correction of the downsampled EEG signal. The specific operation is to remove the channel mean, that is, to subtract the mean of each channel from its signal to eliminate the influence of DC offset and low-frequency drift; S303, apply a bandpass filter to the EEG signal after baseline correction to remove noise interference and obtain a noise-reduced EEG signal. In this embodiment, a 0.5-50Hz bandpass filter is used to effectively filter out high-frequency and low-frequency noise components.

[0024] Because the left and right hemispheres of the human brain show asymmetry in emotional responses, such as Figure 2 As shown, in this embodiment, each source domain S Multiple time subdomains are formed by time scrambling P When including the source domain S The EEG signal feature samples in the image are divided into the left hemisphere and the right hemisphere. For each pair of two EEG signal feature samples at time t and time t+1 that are adjacent in time, the left hemisphere part of the EEG signal feature sample at time t (denoted as L t ), the right hemisphere part of the EEG signal feature sample at time t+1 (denoted as R t+1 ) splicing and combining to obtain the time subdomain P The EEG signal feature sample at time t is the left hemisphere part of the EEG signal feature sample at time t+1 (denoted as L t+1 ), the right hemisphere part of the EEG signal feature sample at time t (denoted as R t ) splicing and combining to obtain the time subdomain P The EEG signal feature sample at time t+1 finally forms the time subdomain P Multiple EEG signal feature samples.

[0025] In this embodiment, each source domain S The EEG signal feature samples are respectively O The EEG signal feature samples are mixed to form multiple mixed sub-domains Q The EEG signal feature samples refer to the EEG signal feature samples in different domains under the same stimulus, which are mixed in equal proportions to participate in the training and source domains. S and target domain O Forming multiple hybrid subdomains QConsidering that the EEG signals of different subjects are synchronized when perceiving the same induced scene, based on the similarity of EEG signals in different domains under the same stimulus, the source domain and the target domain are mixed in equal proportion to participate in the training. That is, the differential entropy features of the source domain subjects and the target domain subjects in each stimulus scene are averaged in the same period to obtain the representative features of the scene and form a new subdomain Q.

[0026] like Figure 3 As shown, the domain adversarial neural network in this embodiment includes: a sample pairing module, which is used to input the source domain S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are respectively O The EEG signal feature samples are combined and paired to form various types of EEG signal feature sample pairs; the feature representation module is used to extract features from the EEG signal feature samples from the source domain or the target domain; the adversarial discriminator is used to determine the domain type of the input; the task classifier is used to determine the task type of the input.

[0027] The sample pairing module will input S , P , Q The data of the source domain and its subdomains ( S , P , Q ) and the target domain ( O ) to form different domain combination types, such as " S and O "," P and O "," Q and O ".like Figure 4 As shown, in this embodiment, the input source domain S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are respectively OWhen the EEG signal feature samples are combined and paired, the multiple types of EEG signal feature sample pairs formed include the following six types of EEG signal feature sample pairs: (1) Same label, same source domain: both samples belong to the same label and both come from the source domain; (2) Different labels, same source domain: both samples belong to different labels and both come from the source domain; (3) Same label, different domain: both samples belong to the same label, but come from the source domain and the target domain respectively; (4) Different labels, different domains: two samples belong to different labels and come from the source domain and the target domain respectively; (5) Same label, same target domain: both samples belong to the same label and both come from the target domain; (6) Different labels, same target domain: two samples belong to different labels and both come from the target domain. Among them, some types involve labeled target domain samples. However, the sub-labels in the target domain are usually unknown. In the study of this embodiment, this embodiment uses a small number of samples of the target object to complete the sample pairing. The number of calibration samples is much smaller than the number of samples included in the source domain. Since there are 3 types of combinations, each of which consists of 6 types of pairings, the example of this embodiment involves a total of 18 types.

[0028] In this embodiment, the feature representation module is a multilayer perceptron MLP with shared parameters, and the multilayer perceptron MLP includes two fully connected hidden layers and an output layer; the adversarial discriminator consists of two fully connected layers and an output layer; the task classifier consists of two fully connected layers and an output layer. The feature representation module uses a multilayer perceptron (MLP) with shared parameters to represent the features of the paired samples, and includes two fully connected hidden layers and an output layer. The first two fully connected layers are used to extract high-dimensional features from the DE features, and the output layer is used to output the feature dimensions to the adversarial discriminator and the task classifier. The adversarial discriminator consists of two fully connected layers, followed by an output layer, which determines the domain type of the input. The task classifier also consists of two fully connected layers, followed by an output layer, which determines the task type of the input. It should be noted that this embodiment only involves the application of the fully connected layer, and does not involve related improvements, so its specific implementation details will not be described in detail here.

[0029] It should be noted that the selection method of the training sample data set is different in cross-subject emotion recognition and cross-session emotion recognition. In the cross-subject EEG analysis experiment, one subject needs to be selected from all the subjects as the target domain, and the rest of the subjects are selected as the source domain. The training set includes a small amount of labeled data in the target domain and labeled data in the source domain. In this embodiment, the source domain S , Time subdomain P , mixed subdomain QBefore the EEG signal feature samples are input into the domain adversarial neural network for training to form branch prediction results, the step of training the domain adversarial neural network is also included, and the loss function used for training and optimizing the network parameters of the branch network is the classification loss and domain loss The sum of the two, classification loss The calculation function expression is: , In the above formula, is the number of samples, is the task label, represents the task classifier, and Indicates task label The feature representation obtained by extracting the two EEG signal feature samples in the corresponding EEG signal feature sample pair by the feature representation module; domain loss The calculation function expression is: , In the above formula, represents the mathematical expectation, is the EEG signal feature sample pair The type tag, represents the adversarial discriminator, For splicing operation, and is the EEG signal feature sample pair The two EEG signal feature samples in are extracted by the feature representation module to obtain the feature representation. After training the branch domain adversarial network, this embodiment uses the target domain EEG data to evaluate the performance of these domain adaptation models to obtain branch prediction results. According to the evaluation results, voting prediction is performed, and the result with the most results is the final prediction result of the target domain O, which is specifically neutral, sad, fearful and happy in this embodiment.

[0030] In order to verify the multi-source domain adversarial EEG signal classification method based on multi-task learning in this embodiment, the public SEED dataset and SEED-IV dataset are used for cross-subject recognition in this embodiment. The SEED dataset records in detail the EEG signals of 15 participants when they are exposed to three types of emotional stimuli (positive, neutral and negative). The EEG signals are recorded at a high sampling rate of 1000Hz through 62 electrodes. Each subject conducts 3 experiments (one week apart), and watches 15 different videos in each experiment. The SEED-IV dataset contains EEG emotional data of 15 participants, which are divided into four states: neutral, sad, fearful and happy. Similar to the SEED dataset, the SEED-IV dataset is recorded at a high sampling rate of 1000Hz through 62 electrodes. Each subject conducts 3 experiments (one week apart), and the stimulation comes from 24 different videos they watch. Experimental results of emotion recognition across subjects: In this embodiment, each subject in each session (the same experiment conducted by all subjects is regarded as one session) is taken as the target domain in turn, and the remaining 14 subjects are taken as the source domain. Classification experiments are conducted on the three sessions respectively. The experimental results of SEED and SEED-IV datasets are shown in Tables 1 and 2 respectively.

[0031] Table 1 Emotion recognition results across subjects based on the SEED dataset (%)

[0032] Table 2 Emotion recognition results across subjects based on SEED-IV dataset (%)

[0033] As shown in Table 1 and Table 2, the average accuracy of the method in this embodiment for three sessions for each subject in the SEED dataset is above 90%, and the average recognition accuracy of the three sessions for each subject in the SEED-IV dataset is above 80%, which fully proves that the method in this embodiment has better generalization ability in the cross-subject emotion recognition task.

[0034] In summary, the multi-source domain adversarial EEG signal feature analysis method based on multi-task learning in this embodiment includes respectively collecting multi-channel EEG signals of corresponding analysis tasks and extracting features; forming multiple time subdomains by time scrambling each source domain; mixing each source domain with the target domain to form multiple mixed subdomains; inputting the corresponding subdomains into the domain adversarial neural network for training for feature analysis; voting prediction of the branch prediction results; and obtaining a multi-source domain adaptive feature analysis system. The method of this embodiment aims at the problem that there are large differences in EEG patterns between different individuals and it is difficult to analyze them. Considering the influence of the interference of task-irrelevant features on the differences of subjects, a variety of task-irrelevant adversarial tasks are introduced to enhance the learning ability of the multi-source domain adversarial model. This embodiment proposes to construct multiple subdomains by time scrambling and domain mixing. These subdomains adopt a domain adversarial neural network structure by providing more diverse training samples, which includes a sample pairing module, a feature representation module, an adversarial discriminator and a task classifier. This architecture jointly trains EEG signals from multiple subdomains so that the network can simultaneously learn task-related features and de-domained features, thereby effectively enhancing the ability to classify EEG signals. The obtained multi-source domain adversarial EEG signal feature analysis method has achieved excellent performance in cross-subject emotion recognition experiments, and has the advantages of high evaluation accuracy and strong generalization performance. The method in this embodiment enhances the learning ability of the multi-source domain adversarial model by introducing a variety of task-independent adversarial tasks, which can improve the robustness and generalization of features and improve the accuracy of EEG signal classification prediction.

[0035] In addition, this embodiment also provides a multi-source domain adversarial EEG signal classification method based on multi-task learning, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning.

[0036] In addition, this embodiment also provides a computer-readable storage medium, which stores a computer program or instruction, and the computer program or instruction is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning through a processor.

[0037] In addition, this embodiment also provides a computer program product, including a computer program or instructions, which are programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning through a processor.

[0038] Those skilled in the art should understand that the technical solutions provided by the embodiments of the present application may be in the form of methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the process Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0039] The above is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions under the concept of the present invention belong to the protection scope of the present invention. It should be pointed out that for ordinary technicians in this technical field, some improvements and modifications without departing from the principle of the present invention should also be regarded as the protection scope of the present invention.

Claims

1. A multi-source domain adversarial EEG signal classification method based on multi-task learning, characterized in that: The following steps are included: Get the target domain respectively O and multiple source domains S EEG signal feature samples; Each source domain S The EEG signal feature samples are respectively scrambled to form time subdomains P Multiple EEG signal feature samples; Each source domain S The EEG signal feature samples are respectively O The EEG signal feature samples are mixed to form multiple mixed sub-domains Q EEG signal feature samples; Source Domain S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are input into the domain adversarial neural network for training to form branch prediction results; the branch prediction results are voted to determine the target domain O The final prediction result.

2. The multi-source domain adversarial EEG signal classification method based on multi-task learning according to claim 1 is characterized in that: Each source domain S Multiple time subdomains are formed by time scrambling P When including the source domain S The EEG signal feature samples in the image are divided into the left hemisphere and the right hemisphere. For each pair of two EEG signal feature samples at time t and time t+1 that are adjacent in time, the left hemisphere part of the EEG signal feature sample at time t and the right hemisphere part of the EEG signal feature sample at time t+1 are spliced ​​and combined to obtain the time subdomain P The EEG signal feature sample at time t is obtained by splicing the left hemisphere part of the EEG signal feature sample at time t+1 and the right hemisphere part of the EEG signal feature sample at time t to obtain the time subdomain P The EEG signal feature sample at time t+1 finally forms the time subdomain P Multiple EEG signal feature samples.

3. The multi-source domain adversarial EEG signal classification method based on multi-task learning according to claim 1 is characterized in that: Each source domain S The EEG signal feature samples are respectively O The EEG signal feature samples are mixed to form multiple mixed sub-domains Q The EEG signal feature samples refer to the EEG signal feature samples in different domains under the same stimulus, which are mixed in equal proportions to participate in the training and source domains. S and target domain O Forming multiple hybrid subdomains Q EEG signal feature samples.

4. The multi-source domain adversarial EEG signal classification method based on multi-task learning according to claim 1 is characterized in that: The domain adversarial neural network includes: a sample pairing module for inputting a source domain S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are respectively O The EEG signal feature samples are combined and paired to form various types of EEG signal feature sample pairs; the feature representation module is used to extract features from the EEG signal feature samples from the source domain or the target domain; the adversarial discriminator is used to determine the domain type of the input; the task classifier is used to determine the task type of the input.

5. The multi-source domain adversarial EEG signal classification method based on multi-task learning according to claim 4 is characterized in that: The source domain to be entered S , Time subdomain P , mixed subdomain Q The EEG signal feature samples are respectively O When the EEG signal feature samples are combined and paired, the various types of EEG signal feature sample pairs formed include the following six types of EEG signal feature sample pairs: (1) Same label, same source domain: both samples belong to the same label and both come from the source domain; (2) Different labels, same source domain: both samples belong to different labels and both come from the source domain; (3) Same label, different domains: both samples belong to the same label but come from the source domain and target domain respectively; (4) Different labels, different domains: two samples belong to different labels and come from the source domain and target domain respectively; (5) Same label, same target domain: both samples belong to the same label and both come from the target domain; (6) Different labels, same target domain: two samples belong to different labels and both come from the target domain.

6. The multi-source domain adversarial EEG signal classification method based on multi-task learning according to claim 4 is characterized in that: The feature representation module is a multilayer perceptron MLP with shared parameters, and the multilayer perceptron MLP includes two fully connected hidden layers and an output layer; the adversarial discriminator consists of two fully connected layers and an output layer; and the task classifier consists of two fully connected layers and an output layer.

7. The multi-source domain adversarial EEG signal classification method based on multi-task learning according to claim 4 is characterized in that: The source domain S , Time subdomain P , mixed subdomain Q Before the EEG signal feature samples are input into the domain adversarial neural network for training to form branch prediction results, the step of training the domain adversarial neural network is also included, and the loss function used for training and optimizing the network parameters of the branch network is the classification loss and domain loss The sum of the two, classification loss The calculation function expression is: ， In the above formula, is the number of samples, is the task label, represents the task classifier, and Indicates task label The feature representation obtained by extracting the two EEG signal feature samples in the corresponding EEG signal feature sample pair by the feature representation module; domain loss The calculation function expression is: ， In the above formula, represents the mathematical expectation, is the EEG signal feature sample pair The type tag, represents the adversarial discriminator, For splicing operation, and is the EEG signal feature sample pair The two EEG signal feature samples in are extracted by the feature representation module to obtain the feature representation.

8. A multi-source domain adversarial EEG signal classification method based on multi-task learning, comprising a microprocessor and a memory connected to each other, characterized in that: The microprocessor is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning as described in any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program or instruction stored therein, characterized in that: The computer program or instruction is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning described in any one of claims 1 to 7 through a processor.

10. A computer program product comprising a computer program or instructions, characterized in that The computer program or instruction is programmed or configured to execute the multi-source domain adversarial EEG signal classification method based on multi-task learning described in any one of claims 1 to 7 through a processor.

Images