Electroencephalogram signal field adaptation method and system based on cross attention

Abstract

The invention discloses an electroencephalogram signal field adaptation method and system based on cross attention. The method comprises the steps that tested electroencephalogram signal data samples in a target domain and a plurality of source domains are obtained respectively; adopting a source domain selector to select an optimal source domain set for the tested electroencephalogram signal data sample in the target domain; inputting a global cross attention aligner, respectively extracting shallow layer features by using a feature extraction network therein, and guiding by a global adaptation module to realize conditional alignment of the two; taking the shallow layer features after the condition alignment as input of a second classifier and a discriminator, and guiding domain edge distribution alignment through an adversarial alternate training discriminator and a global cross attention aligner; and the network parameters of the freezing discriminator and the global cross attention aligner train a second classifier for performing electroencephalogram signal classification according to the input electroencephalogram signal data. The method aims at improving the accuracy and generalization performance of cross-domain electroencephalogram signal cognitive load evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram (EEG) signal analysis, and particularly relates to a method and system for domain adaptation of EEG signals based on cross-attention. Background Art

[0002] In recent years, significant progress has been made in feature extraction and classification based on electroencephalogram (EEG). However, due to the significant individual differences in EEG signals, feature extraction and classification in the case of cross-subject still face challenges. Traditional cross-subject EEG signal classification methods often require data re-acquisition and calibration for each new subject, which are time-consuming, laborious, and costly. Therefore, cross-subject EEG signal classification with few labels is a key issue. Currently, some studies focus on the obvious and stable invariant features between different subjects, and use machine learning methods to find obvious and shared features to construct cross-subject models. There are also many studies using transfer learning methods to explore the distribution alignment between the source domain and the target domain. Some methods use statistical metrics to reduce the domain difference, such as the maximum mean difference (MMD) and correlation alignment. Inspired by the generative adversarial network (GAN), adversarial learning-based methods use discriminators to automatically reduce the distribution difference and learn domain-invariant features. However, these domain adaptation methods align the distribution statistics by constraining the features or the similarity of features, and tend to smooth the feature gully to find the shared feature space, which will cause the loss of shallow features in the training of deep domain adaptation networks and cannot effectively solve the contradiction between "distribution alignment and feature retention", thus affecting the classification accuracy. Therefore, how to overcome these challenges and further improve the cross-subject classification performance based on EEG signals has become a key technical problem to be solved urgently. Summary of the Invention

[0003] The technical problem to be solved by the present invention: Aiming at the above problems of the prior art, the present invention provides a method and system for domain adaptation of EEG signals based on cross-attention. The present invention aims to solve the problems of feature alignment and inter-group specificity protection in the cross-subject cognitive load assessment of EEG signals, and improve the accuracy and generalization performance of cross-domain cognitive load assessment of EEG signals.

[0004] In order to solve the above technical problems, the technical solution adopted by the present invention is as follows: A method for domain adaptation of EEG signals based on cross-attention, comprising the following steps: S101, respectively obtain the EEG signal data samples of the subjects in the target domain and multiple source domains; S102, use a source domain selector including a first classifier to select the k most similar subjects' EEG signal data samples in multiple source domains for each subject's EEG signal data sample in the target domain to form an optimal source domain set; S103. Input the EEG signal data samples of the subjects in each target domain and their corresponding optimal source domain sets into the global cross-attention aligner composed of a feature extraction network and a global adaptation module based on the cross-attention mechanism. Use the feature extraction network to extract the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set respectively, and use the global adaptation module to guide the conditional alignment of the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set; S104. Use the conditionally aligned shallow features as the inputs of the second classifier and the discriminator, and guide the alignment of the domain marginal distributions through adversarial alternating training of the discriminator and the global cross-attention aligner; S105. Freeze the network parameters of the discriminator and the global cross-attention aligner and train the second classifier to classify EEG signals according to the input EEG signal data.

[0005] Optionally, the source domain selector in step S102 is composed of an encoder and a first classifier. Both the encoder and the first classifier are composed of three fully connected layers, and each source domain uses a separately trained source domain selector with non-shared parameters.

[0006] Optionally, the feature extraction network in step S103 is a three-cascaded symmetric dual-path network. The three-cascaded symmetric dual-path network includes two paths corresponding to the source domain and the target domain respectively. The sizes of the EEG signal data samples input into the two paths for the source domain and the target domain are the same. Each path includes an input layer, a frequency-domain convolutional layer, a time-domain convolutional layer, a channel convolutional layer, an average pooling layer, and a projection layer connected in sequence. The projection layer is used to project the extracted features into an embedded sequence with a single channel and the same dimension suitable for processing based on the attention mechanism as the shallow features finally obtained by the feature extraction network.

[0007] Optionally, guiding the conditional alignment of the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set in step S103 includes: S201. Divide the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of the EEG signal data samples in the optimal source domain set into multiple slices of the same length respectively, and use a linear layer to generate a key matrix K, a query matrix Q, and a value matrix V. Among them, the key matrix K and the query matrix Q come from the shallow features of the EEG signal data samples in the optimal source domain set, and the value matrix V comes from the shallow features of the EEG signal data samples of the subjects in the target domain; S202. Execute attention queries through multiple attention heads according to the following formula: , , wherein, is the attention query result of the th attention head, is the attention query, is the EEG signal data sample in the source domain, is the EEG signal data sample in the target domain, , and are the weight matrices of the query, key and value of the th attention head respectively, is the dimension of the key matrix K; S203, the shallow features after conditional alignment are obtained by concatenating the attention query results of each attention head through the following formula: , wherein, is the shallow feature obtained by concatenating the attention query results of each attention head after conditional alignment, ~ are the attention query results of the 1st to hth attention heads respectively.

[0008] Optionally, before inputting the EEG signal data sample of each subject in the target domain and its corresponding optimal source domain set into the global cross-attention aligner composed of a feature extraction network and a global adaptation module based on the cross-attention mechanism in step S103, it further includes enhancing the EEG signal data sample of each subject in the target domain: generating weight values for the EEG signal data sample of each subject in the target domain and normalizing them, randomly selecting two EEG signal data samples, using the normalized weight values to perform weighted combination to generate a new EEG signal data sample, and the label of the EEG signal data sample adopts the label of the original EEG signal data sample with a larger normalized weight value.

[0009] Optionally, when guiding the domain marginal distribution alignment by adversarial alternating training of the discriminator and the global cross-attention aligner in step S104, it includes using a gradient reversal layer GRL to force the feature distributions to converge, and the function expression of the adopted loss function is: , , wherein, is the loss function adopted by the adversarial alternating training of the discriminator and the global cross-attention aligner, represents that the EEG signal data sample comes from the source domain , represents that the EEG signal data sample from the target domain , respectively represent the EEG signal data samples from the random interpolation domain , is the discriminator, is the cross-attention mechanism, is the feature extraction network for the shallow features extracted by is the feature extraction network for the shallow features extracted by is the hyperparameter for controlling the penalty intensity, is the gradient penalty calculated by the gradient reversal layer GRL for and is the gradient operator, the samples of the random interpolation domain of are constructed as follows:

[0010] wherein, is the random weight, and there is .

[0011] Optionally, when training the second classifier by freezing the network parameters of the discriminator and the global cross-attention aligner in step S105, the functional expression of the loss function used is: , , , wherein, is the loss function used when training the second classifier by freezing the network parameters of the discriminator and the global cross-attention aligner, is the classification loss, is the adversarial loss, and respectively represent the numbers of EEG signal data samples in the source domain and the target domain, is the number of EEG signal categories, is the label that the i-th EEG signal data sample belongs to the EEG signal category c, is the predicted value that the second classifier predicts that the i-th EEG signal data sample belongs to the EEG signal category c, represents the EEG signal data sample from the source domain , is the discriminator, is the cross-attention mechanism, is the feature extraction network For the shallow features obtained by extraction.

[0012] In addition, the present invention also provides an electroencephalogram (EEG) signal domain adaptation system based on cross-attention, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the EEG signal domain adaptation method based on cross-attention.

[0013] In addition, the present invention also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the EEG signal domain adaptation method based on cross-attention through a processor.

[0014] In addition, the present invention also provides a computer program product, including a computer program or instruction, and the computer program or instruction is programmed or configured to execute the EEG signal domain adaptation method based on cross-attention through a processor.

[0015] Compared with the prior art, the present invention can mainly achieve the following beneficial effects: The EEG signal domain adaptation method based on cross-attention of the present invention includes obtaining EEG signal features of a target domain and multiple source domains; calculating the similarity between the target domain and each source domain through a source domain selector, screening out the Top-k source domains and generating weighted enhanced target domain features; using a global cross-attention aligner (GCA) to extract spatio-temporal features through a multi-stage cascaded symmetric dual-path network, while achieving the alignment of the conditional distributions of the source domain and the target domain, retaining key shallow features through the cross-attention mechanism and the residual structure; performing adversarial training using a domain discriminator to align the marginal distributions; and finally jointly optimizing a classifier to achieve high-precision evaluation of cross-subject cognitive load. The EEG signal domain adaptation method based on cross-attention of the present invention can solve the problems of feature alignment and inter-group specificity protection in the evaluation of cross-subject cognitive load of EEG signals in the prior art, and improve the accuracy and generalization performance of cross-domain EEG signal cognitive load evaluation. Aiming at the problem that it is difficult to balance between cross-subject EEG signal feature alignment and preservation, the present invention designs a global cross-attention aligner based on cross-attention, and integrates it with a discriminator, a classifier and a source domain selector into a complete cross-subject EEG signal classification algorithm, which can solve the problems of feature alignment and inter-group specificity protection in cross-subject EEG signals, and has the advantages of high recognition accuracy and strong generalization performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 It is a schematic diagram of the basic process of the method in the embodiment of the present invention.

[0017] Figure 2 It is a schematic diagram of the working principle of the source domain selector in the embodiment of the present invention.

[0018] Figure 3 It is a schematic diagram of the network structure of the global cross-attention aligner and its backend part in the embodiment of the present invention.

[0019] Figure 4 It is a schematic diagram of the network structure of the feature extraction network in the embodiment of the present invention. Detailed implementation manners

[0020] In order to enable those skilled in the art to better understand the technical solution of the present invention, the technical solution of the present invention will be further described in detail below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0021] As Figure 1 shown, the EEG signal domain adaptation method based on cross-attention in this embodiment includes the following steps: S101, respectively obtain the EEG signal data samples of the subjects in the target domain and multiple source domains; S102, use the source domain selector including the first classifier to select the k most similar EEG signal data samples of the subjects in each target domain from the EEG signal data samples of the subjects in multiple source domains to form an optimal source domain set; S103, input the EEG signal data samples of the subjects in each target domain and their corresponding optimal source domain sets into the global cross-attention aligner composed of a feature extraction network and a global adaptation module based on the cross-attention mechanism, use the feature extraction network to respectively extract the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set, and use the global adaptation module to guide the conditional alignment of the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set; S104, use the conditionally aligned shallow features as the inputs of the second classifier and the discriminator, and guide the alignment of the domain marginal distributions by alternately training the discriminator and the global cross-attention aligner; S105, freeze the network parameters of the discriminator and the global cross-attention aligner and train the second classifier to be used for EEG signal classification according to the input EEG signal data.

[0022] Figure 2 It is a schematic diagram of the working principle of the source domain selector in this embodiment, Figure 2The number of subjects in the target domain is m (subject 1 to subject m). After the source domain selector selects the electroencephalogram (EEG) signal data samples of the k subjects (subject 1 to subject k) most similar to the EEG signal data sample of each subject in the target domain from multiple source domains to form the optimal source domain set, finally m optimal source domain sets are screened out, which altogether contain the EEG signal data samples of m×k subjects in the source domain. The EEG signal data sample in step S101 of this embodiment refers to the EEG signal containing multiple channels induced by a subject completing a single cognitive load task experiment. When a subject performs a specific task, the cerebral cortex is stimulated by the external environment and will generate associated evoked electrical activities. In the method of this embodiment, the objective physical signal of EEG is selected to realize cognitive load assessment, which can alleviate the subjective influence of interference tasks brought by traditional assessment methods such as self-administered questionnaires of subjects, and effectively improve the accuracy and generalization ability of cognitive load assessment.

[0023] As an alternative implementation, the acquisition of the features of the EEG signal in this embodiment includes: denoising the EEG signals of multiple channels of each subject; dividing the denoised EEG signals into time windows according to a fixed-size time window; calculating the power spectral density (PSD) of each electrode in the 1-100 Hz frequency band for the EEG data in each time window and normalizing it. Specifically, the EEG frequency domain features of different channels are obtained through fast Fourier transform. The frequency bands include δ (1-3 Hz), θ (4-7 Hz), α (8-13 Hz), β (13-30 Hz), and γ (31-50 Hz). Therefore, we obtain the frequency domain features within sequential time. The time-frequency data is used as the input of the subsequent model, and a very small number of target domains are selected as the labeled data to input into the model for training. Among them, denoising can adopt a feasible denoising method according to needs. For example, as an alternative implementation, the denoising in this embodiment includes: performing band-pass filtering on the EEG data to remove low-frequency drift and high-frequency noise. As an alternative implementation, in this embodiment, the original EEG signal is processed by a 0.5-50 Hz band-pass filter to remove noise interference; the myoelectric, electrooculogram, and electrocardiogram artifacts are removed from the filtered signal. The myoelectric artifacts are detected and removed through the built-in function of MNE, and independent component analysis (ICA) is used to remove the electrooculogram (EOG) and electrocardiogram (ECG) artifacts; the data is downsampled.

[0024] The source domain selector is a classifier. A small amount of labeled target domain data is used to test each of the other trained source domain models, and the k source domains with the highest accuracy are selected as the subsequent source domain training set L to make the best use of the positive source domain data as much as possible. The source domain selector is a classifier. Specifically, the required network model can be adopted according to needs, such as Figure 2As shown, as an alternative embodiment, the source domain selector in step S102 of this embodiment is composed of an encoder and a first classifier. Both the encoder and the first classifier are composed of three fully connected layers. The last fully connected layer is used to reduce the feature dimension to the corresponding cognitive load level as the prediction result.

[0025] To achieve selection for different cross-domains, in this embodiment, each source domain uses a separately trained source domain selector with non-shared parameters. The parameters of each source domain model are not shared and are trained separately. The loss function used to train and optimize the network parameters is denoted as and its calculation function expression is: , where N is the total number of EEG signal data samples for each subject, represents each EEG signal data sample taken from subject S x , and are the prediction result and the true label of the classifier for the EEG signal data sample x respectively. is the cross-entropy loss calculation, and its calculation function expression is: , where is the true label, is the prediction result.

[0026] Figure 3 is the network structure schematic diagram of the global cross-attention aligner and its backend part in this embodiment. In this embodiment, the global cross-attention aligner is used to maximize the retention of shallow features while aligning the conditional distributions. The global cross-attention aligner is composed of a feature extraction network and a global adaptation module based on the cross-attention mechanism. The feature extraction network is used to extract the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set respectively. The global adaptation module is used to guide the conditional alignment of the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set. Conditional alignment means making the feature distributions of the source domain and the target domain as consistent as possible under the given class labels in transfer learning. Its core is to reduce the inter-class distribution differences of cross-domain data and avoid the performance degradation of the classifier due to domain shift.

[0027] The three-cascade symmetric dual-path network receives the input of two data with the same size as the initial feature extractor, capturing the inherent temporal dynamics and spatial dependencies in the EEG signal. As Figure 4As shown in the figure, in step S103 of this embodiment, the feature extraction network is a three - cascaded symmetric dual - path network. The three - cascaded symmetric dual - path network includes two paths corresponding to the source domain and the target domain respectively. The sizes of the EEG signal data samples input to the two paths are the same. Each path includes an input layer, a frequency - domain convolutional layer, a time - domain convolutional layer, a channel convolutional layer, an average pooling layer, and a projection layer connected in sequence. The projection layer is composed of a 1×1 convolution and a vector transformation module, which is used to project the extracted features into an embedded sequence with a single channel and the same dimension suitable for processing based on the attention mechanism as the shallow features finally obtained by the feature extraction network, and serve as the input of the subsequent global adaptation module. See Figure 4 , the three - cascaded symmetric dual - path network acts as an initial feature extractor to receive the input of two data with the same size, capture the inherent temporal dynamics and spatial dependencies in the EEG signals. The three - cascaded symmetric dual - path network adopts a symmetric dual - path convolution design to alternately model the time - to - space and space - to - time relationships, thus realizing rich feature encoding across domains. This cascaded structure enhances the model's ability to retain domain - specific nuances while improving the generalizable representation. After a series of convolution, normalization, and pooling operations, the extracted features are projected into an embedded sequence with a single channel and the same dimension suitable for processing based on the attention mechanism.

[0028] The global adaptation module is used to balance the realization of shallow feature protection and domain - conditional distribution alignment. The cross - attention mechanism allows a small number of labeled target samples to guide the source - domain representation through dynamic attention weights, realizing effective domain adaptation and feature protection. In step S103 of this embodiment, the conditional alignment of the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set by using the global adaptation module includes: S201, respectively divide the shallow features of the EEG signal data samples of the subjects in the target domain and the shallow features of the EEG signal data samples in the optimal source domain set into multiple slices of the same length, and use a linear layer to generate a key matrix K, a query matrix Q, and a value matrix V. Among them, the key matrix K and the query matrix Q come from the shallow features of the EEG signal data samples in the optimal source domain set, and the value matrix V comes from the shallow features of the EEG signal data samples of the subjects in the target domain; S202, perform attention queries according to the following formula through multiple attention heads: , , where, is the attention query result of the th attention head, is the attention query, is the EEG signal data sample in the source domain, is the EEG signal data sample in the target domain, , and are the weight matrices of the query, key, and value of the th attention head respectively (used by the linear layer to generate the key matrix K, query matrix Q, and value matrix V), is the dimension of the key matrix K; S203, the attention query results of each attention head are concatenated through the following formula to obtain the shallow features after conditional alignment: , where is the shallow feature obtained by concatenating the attention query results of each attention head after conditional alignment, ~ are the attention query results of the 1st to hth attention heads respectively. As an optional implementation, a linear layer can be added at the end to improve the fitting ability.

[0029] To ensure the balance of feature representation, in step S103 of this embodiment, a weighted average strategy is adopted to enhance a small amount of labeled target domain data to the same sample dimension as the source domain. Specifically, before inputting each EEG signal data sample of the subjects in the target domain and its corresponding optimal source domain set into the global cross-attention aligner composed of a feature extraction network and a global adaptation module based on the cross-attention mechanism, this embodiment also includes enhancing each EEG signal data sample of the subjects in the target domain: generating weight values for each EEG signal data sample of the subjects in the target domain and normalizing them, randomly selecting two EEG signal data samples, using the normalized weight values for weighted combination to generate new EEG signal data samples, and the label of the EEG signal data sample adopts the label of the original EEG signal data sample with a larger normalized weight value. As Figure 2 shown, the source domain consists of all the k selected source domains. For each sample in the small labeled target set, a weight value is randomly generated for each sample and normalized. Randomly select two samples, use the normalized random weights for weighted combination, and determine the label according to the weights. This enhancement method effectively enhances the representativeness of the target subset and ensures that it can be fairly compared with the source samples for subsequent feature alignment. The enhanced part of the target domain data and the source domain data are simultaneously input into the global cross-attention aligner.

[0030] This embodiment also includes achieving the marginal distribution alignment of source domain and target domain features by alternately training the global attention domain aligner composed of the feature extractor and multi-head attention in S103 and the domain discriminator in S104 based on the core idea of the generative adversarial network. The aligner is trained to maximize the error of the discriminator, and the discriminator is trained to minimize the classification error between the source domain and target domain features, and they are alternately updated until a Nash equilibrium is reached. The domain discriminator in step S104 of this embodiment is used for adversarial training to align the marginal distributions of the source domain and target domain. The domain discriminator in S104 includes two fully connected layers, and the second fully connected layer is used to classify the features into the source domain and target domain.

[0031] When guiding the alignment of domain marginal distributions by alternately training the discriminator and the global cross-attention aligner through adversarial training, the alignment of domain marginal distributions refers to aligning the overall feature distributions of the source domain and target domain, ignoring category information. Its purpose is to eliminate the global differences between domains (such as device differences and inter-subject variations). Through the adversarial game between the discriminator and the feature generator (global adaptation module), the feature extractor is forced to generate domain-invariant features. To ensure the stability of the adversarial learning process, when guiding the alignment of domain marginal distributions by alternately training the discriminator and the global cross-attention aligner in step S104 of this embodiment, it includes using a gradient reversal layer GRL to force the feature distributions to converge, and the functional expression of the loss function adopted is: , , where, is the loss function adopted for alternately training the discriminator and the global cross-attention aligner through adversarial training, represents the electroencephalogram signal data sample from the source domain , represents the electroencephalogram signal data sample from the target domain , respectively represent the electroencephalogram signal data sample from the randomly interpolated domain , is the discriminator, is the cross-attention mechanism, is the feature extraction network for the shallow features extracted by is the feature extraction network for the shallow features extracted by is the hyperparameter for controlling the penalty strength, is the gradient penalty calculated by the gradient reversal layer GRL for ​is a gradient operator, a random interpolation value range samples is constructed as follows:

[0032] wherein, is a random weight, and there is .

[0033] In this embodiment, in step S105, after freezing the aligner and the domain discriminator, a classifier (the second classifier) is separately trained to further extract features and classify EEG signals in the alignment space. The second classifier in step S105 includes three fully connected layers. The first two layers are used to extract domain-invariant features of the alignment signal and reduce the dimension, and the second layer is used to reduce the dimension to the number of categories. It should be noted that only the applications of the fully connected layer and the convolutional layer are involved in this embodiment, and no related improvements are involved, so the specific implementation details thereof will not be elaborated herein. When training the second classifier by freezing the network parameters of the discriminator and the global cross-attention aligner in step S105 of this embodiment, the function expression of the loss function used is: , , , wherein, is the loss function used when training the second classifier by freezing the network parameters of the discriminator and the global cross-attention aligner, is the classification loss, is the adversarial loss (the purpose is to ensure domain adaptability and always confuse the discriminator), and are the numbers of EEG signal data samples in the source domain and the target domain respectively, is the number of EEG signal categories, is the label that the i-th EEG signal data sample belongs to the EEG signal category c, is the predicted value of the second classifier for the i-th EEG signal data sample belonging to the EEG signal category c, represents the EEG signal data sample comes from the source domain , is the discriminator, is the cross-attention mechanism, is the feature extraction network for the shallow features extracted by

[0034] To verify the EEG signal domain adaptation method based on cross-attention in this embodiment, in this embodiment, the EEG signal data of the N-back task and the MATB-II task from the publicly available COG-BCI dataset and the EEG signal data of the mental arithmetic task from the Open Access Dataset for EEG+NIRS Single-Trial Classification are used to test their classification performance and generalization ability in the cross-subject cognitive load assessment task.

[0035] The COG-BCI dataset is a publicly available multi-task EEG dataset designed to study EEG signal characteristics under different cognitive loads. In this dataset, the N-Back task is used to induce low, medium, and high levels of cognitive load through different difficulty levels (0-Back, 1-Back, and 2-Back). Each task consists of three blocks, and each block contains 48 trials. In each trial, a number (1 to 9) is presented for 500 ms first, followed by a blank screen for 1500 ms. Participants are required to press the space bar when the current number matches the number n positions before. Each block lasts about 2 minutes, and participants need to complete 3 blocks for each difficulty level (a total of 9 blocks). The number of hits for all trials is fixed at 16 (16 hit trials per block). The EEG signals and task performance of 29 healthy participants were recorded throughout the process. The MATB-II task is a multi-task operating environment developed by NASA, designed to simulate real cognitive load scenarios. Four subtasks of MATB-II are used in the COG-BCI dataset: System Monitoring (SYSMON), Tracking Task (TRACK), Communication Task (COMM), and Resource Management Task (RESMAN). Each MATB-II task lasts about 5 minutes, and participants need to complete three independent task runs, corresponding to low, medium, and high difficulties respectively. Under the low-difficulty condition, participants only need to complete the system monitoring and tracking tasks; under the medium-difficulty condition, the resource management task is added; under the high-difficulty condition, the communication task is further added, and the difficulty of the tracking task is increased. The EEG signals and task performance of 29 healthy participants were recorded throughout the process. For all tasks in the COG-BCI dataset, data acquisition was performed using 64-channel active Ag-AgCl electrodes (ActiCap, Brain Products GmbH), arranged according to the international 10-20 system, with a sampling rate of 500 Hz, and the raw data was not filtered. All tasks were verified at the subjective, behavioral, and physiological levels, ensuring their practicality in brain-computer interface research.

[0036] The mental arithmetic dataset contains EEG signal recordings of 29 healthy adults during mental arithmetic tasks (MA) and baseline tasks, which can be used for binary classification research on cognitive load. At the start of the mental arithmetic task, an initial subtraction formula (such as "384 - 8") is presented on the screen for 2 seconds. Participants need to remember the initial numbers and continuously subtract a single digit from the result of the previous subtraction during the task. During the task, only a black fixation cross is shown on the screen, and participants need to perform consecutive subtraction operations mentally. The task lasts for 10 seconds. During the baseline task, only a black fixation cross is shown on the screen, and participants are required to stay relaxed and not perform any cognitive activities. The task lasts for 10 seconds. A random rest period of 15 to 17 seconds is set between each task cycle to reduce fatigue effects. The mental arithmetic and baseline tasks are randomly alternated in the experiment, and each task is repeated 20 times. The EEG data is collected through a 32-channel EEG cap with a sampling rate of 250 Hz.

[0037] Results of the cross-subject cognitive load assessment experiment: In this embodiment, each subject is taken as the target domain in turn, and the remaining 28 subjects are taken as the source domain, and the average accuracy of the 29 subjects is used as the cross-subject cognitive load assessment result. The results are shown in Table 1.

[0038] Table 1 Cross-subject cognitive load assessment results (%)

[0039] Referring to Table 1, it can be seen that the average recognition accuracy of each subject in the three datasets by the method of this embodiment is above 82%, the minimum is 82.53%, and the maximum is 1%. This fully proves that the method of this embodiment has better generalization ability in the cross-subject cognitive load assessment task.

[0040] To further verify the performance of the deep learning model of the method of this embodiment in cross-subject tasks, it is compared with currently relatively advanced existing methods for comparison. The existing methods for comparison include: DeepConvNet, EEGNeX, ATCNet, MSTNet, CNN + LSTM, swin Transformer. These existing methods for comparison propose different network architectures and strategies that can be used for EEG cognitive load recognition. The results calculated using them on the MATB and N-back task datasets and the mental arithmetic dataset mentioned above are compared with the cross-subject task results of the method of this embodiment, as shown in Tables 2, 3, and 4.

[0041] Table 2 Comparison of cross-subject task results between the method of this embodiment and other existing methods on the MATB dataset

[0042] Table 3 Comparison of cross-subject task results between the method of this embodiment and other existing methods on the N-back dataset

[0043] Table 4 Comparison of cross-subject task results between the method of this embodiment and other existing methods on the mental arithmetic dataset

[0044] As can be seen from Table 2, Table 3 and Table 4, the method of this embodiment is superior to other existing method studies in terms of the accuracy index, and the comparison results prove the effectiveness of the method of this embodiment in cross-subject tasks.

[0045] In addition, this embodiment also verifies the influence of different components on the final performance through ablation experiments. To this end, we studied the effectiveness of removing the multi-head attention, discriminator and adversarial loss from the entire model respectively to evaluate these key components in cross-subject feature extraction. The experimental results on the three datasets are shown in Table 5.

[0046] Table 5 Ablation study based on cross-subject cognitive load assessment

[0047] See Table 5. The ablation experiment in this embodiment is analyzed as follows: 1) The domain discriminator further aligns the marginal distributions of the source domain and the target domain through adversarial training. The results show that when the discriminator is removed, the average classification accuracies of the algorithm on the MATB, N-back, and MA datasets decrease by 4.43%, 6.51%, and 18.36% respectively, while the variances of the classification accuracies increase by 3.21%, 6.91%, and 10.74% respectively, proving that the discriminator module also improves the stability of classification. 2) The multi-head attention part is the most crucial part of the global cross-attention aligner framework. It preserves the underlying features by focusing on weights and residual connections, and at the same time re-weights the target domain with the source domain attention scores to align the conditional distributions between domains, thus balancing the protection of shallow features and domain adaptation. When we remove the multi-head attention part, the average classification accuracies on the MATB, N-back, and MA datasets decrease by 1.93%, 1.75%, and 1.25% respectively. At the same time, the variances of the classification accuracies increase by 0.91%, 0.44%, and 0.41% respectively, proving that it also improves the stability of classification. 3) To ensure domain adaptability, during the classifier training phase, the trained domain discriminator is used to evaluate the correct rate of classifying the source domain so that it can always be confused. In this embodiment, an ablation verification is performed on whether the adversarial loss of this part can maintain domain adaptability. The results show that compared with the case where the adversarial loss is added, the average classification accuracies on the three datasets decrease by 0.6%, 0.76%, and 0.91% respectively. In terms of variance, it decreases by 0.23% on the MATB dataset and increases by 0.02% and 0.35% on the N-back and MA datasets respectively. Therefore, the addition of the adversarial loss does not play a significant role in improving the classification stability, but it can improve the average classification accuracy to a certain extent, proving that it can play a role in maintaining domain adaptability.

[0048] In summary, the EEG signal domain adaptation method based on cross-attention in this embodiment includes collecting multi-channel signals for corresponding analysis tasks; preprocessing the signals, extracting signal features, obtaining a selected source domain sample set by using the similarity metric between samples, and generating weighted enhanced target domain features; based on the feature extraction - multi-head attention network, that is, the global cross-attention aligner to achieve conditional alignment and retain key shallow features; using a domain discriminator for adversarial training to align the marginal distributions; separately optimizing the classifier to achieve domain adaptation feature extraction and analysis of cross-subject EEG signals, and achieving high-precision evaluation on the cognitive load EEG dataset; obtaining an EEG signal domain adaptation system based on cross-attention. The method of this embodiment aims at the problem that there are large differences in EEG patterns between different individuals and it is impossible to balance domain adaptation and specificity retention, and proposes a new multi-source domain adaptation framework called the global cross-attention aligner. This framework consists of a source domain selector, a cascaded dual-path feature extractor with a cross-attention module, a domain discriminator, and a final classifier. This embodiment utilizes the characteristic that samples with high similarity between the source domain subjects and the target domain subjects may have the same feature pattern, and uses the source domain selector to adaptively identify the list of source domains most relevant to a given target domain to improve the efficiency of transfer learning; the aligner extracts the source domain and target domain features through a shallow network respectively, and captures the local and global dependencies of the EEG signals while retaining the shallow discriminative features through the cross-attention mechanism, guiding domain conditional alignment; building a domain discriminant network to alternately train with the domain aligner network through adversarial learning to align the marginal distributions; the classifier is trained on the aligned feature space to achieve robust cross-subject classification. The obtained EEG signal domain adaptation system based on cross-attention has achieved better performance in the experiments of cross-subject EEG cognitive load assessment, and has the advantages of high evaluation accuracy and strong generalization performance.

[0049] In addition, this embodiment also provides an EEG signal domain adaptation system based on cross-attention, including a microprocessor and a memory connected to each other, and the microprocessor is programmed or configured to execute the EEG signal domain adaptation method based on cross-attention.

[0050] In addition, this embodiment also provides a computer-readable storage medium, in which a computer program or instruction is stored, and the computer program or instruction is programmed or configured to execute the EEG signal domain adaptation method based on cross-attention through a processor.

[0051] In addition, this embodiment also provides a computer program product, including a computer program or instruction, and the computer program or instruction is programmed or configured to execute the EEG signal domain adaptation method based on cross-attention through a processor.

[0052] Those skilled in the art should understand that the technical solutions provided by the present invention can be in the form of a method, a system, or a computer program product. Therefore, the present invention can be implemented in the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can be in the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code. The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for realizing the functions specified in one block or multiple blocks. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means realizes the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for realizing the functions specified in one block or multiple blocks. These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for realizing the functions specified in one block or multiple blocks.

[0053] The above is only the preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiments. All technical solutions falling within the idea of the present invention belong to the protection scope of the present invention. It should be noted that for those of ordinary skill in the art in this technical field, several improvements and refinements made without departing from the principle of the present invention should also be regarded as within the protection scope of the present invention.

Claims

1. A cross-attention-based domain adaptation method for electroencephalogram signals, characterized in that Including the following steps: S101, respectively obtain the electroencephalogram (EEG) signal data samples of the subjects in the target domain and multiple source domains; S102, use a source domain selector including a first classifier to select the EEG signal data samples of the k most similar subjects in multiple source domains for the EEG signal data sample of each subject in the target domain to form an optimal source domain set; S103, input the EEG signal data sample of each subject in the target domain and its corresponding optimal source domain set into a global cross-attention aligner composed of a feature extraction network and a global adaptation module based on a cross-attention mechanism. Use the feature extraction network to extract the shallow features of the EEG signal data sample of the subject in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set respectively, and use the global adaptation module to guide the conditional alignment of the shallow features of the EEG signal data sample of the subject in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set; S104, use the conditionally aligned shallow features as the input of the second classifier and the discriminator, and guide the alignment of the domain marginal distributions by alternately training the discriminator and the global cross-attention aligner; S105, freeze the network parameters of the discriminator and the global cross-attention aligner and train the second classifier to be used for EEG signal classification according to the input EEG signal data.

2. The method for adapting electroencephalogram signal fields based on cross-attention according to claim 1, wherein The source domain selector in step S102 is composed of an encoder and a first classifier. Both the encoder and the first classifier are composed of three fully connected layers, and each source domain uses a separately trained source domain selector with non-shared parameters.

3. The method for adapting electroencephalogram signal fields based on cross-attention according to claim 1, characterized in that The feature extraction network in step S103 is a three-cascaded symmetric dual-path network. The three-cascaded symmetric dual-path network includes two paths corresponding to the source domain and the target domain respectively. The sizes of the EEG signal data samples of the source domain and the target domain input by the two paths are the same. Each path includes an input layer, a frequency-domain convolutional layer, a time-domain convolutional layer, a channel convolutional layer, an average pooling layer, and a projection layer connected in sequence. The projection layer is used to project the extracted features into an embedded sequence with a single channel and the same dimension suitable for processing based on the attention mechanism as the shallow features finally obtained by the feature extraction network.

4. The method for adapting electroencephalogram signal fields based on cross-attention according to claim 1, characterized in that, The use of the global adaptation module in step S103 to guide the conditional alignment of the shallow features of the EEG signal data sample of the subject in the target domain and the shallow features of each EEG signal data sample in the optimal source domain set includes: S201, respectively segment the shallow features of the EEG signal data sample of the subject in the target domain and the shallow features of the EEG signal data sample in the optimal source domain set into multiple slices of the same length, and use a linear layer to generate a key matrix K, a query matrix Q, and a value matrix V. Among them, the key matrix K and the query matrix Q come from the shallow features of the EEG signal data sample in the optimal source domain set, and the value matrix V comes from the shallow features of the EEG signal data sample of the subject in the target domain; S202, perform attention queries by multiple attention heads according to the following formula: ， ， Among them, is the attention query result of the -th attention head, is the attention query, is the EEG signal data sample in the source domain, is the EEG signal data sample in the target domain, , and are the weight matrices of the query, key and value of the -th attention head respectively, is the dimension of the key matrix K; S203, splice the attention query results of each attention head according to the following formula to obtain the conditionally aligned shallow features: ， Among them, to splice the attention query results of each attention head to obtain the shallow features after conditional alignment, ~ are the attention query results of the 1st to h-th attention heads respectively.

5. The method for adapting an electroencephalogram signal field based on cross-attention according to claim 1, wherein Before inputting the EEG signal data samples of the subjects in each target domain and their corresponding optimal source domain sets into the global cross-attention aligner composed of a feature extraction network and a global adaptation module based on the cross-attention mechanism in step S103, it further includes enhancing the EEG signal data samples of the subjects in each target domain: generating weight values for the EEG signal data samples of the subjects in each target domain and normalizing them, randomly selecting two EEG signal data samples, and using the normalized weight values to weighted-combine to generate new EEG signal data samples, and the label of the EEG signal data sample adopts the label of the original EEG signal data sample with a larger normalized weight value.

6. The method for adapting an electroencephalogram signal domain based on cross-attention according to claim 1, characterized in that When guiding the alignment of the domain marginal distributions by adversarially and alternately training the discriminator and the global cross-attention aligner in step S104, it includes using a gradient reversal layer GRL to force the feature distributions to converge, and the functional expression of the loss function used is: ， ， Among them, is the loss function adopted for adversarial alternating training of the discriminator and the global cross-attention aligner. represents an EEG signal data sample from the source domain , represents an EEG signal data sample from the target domain , respectively represent EEG signal data samples from the random interpolation domain , is the discriminator, is the cross-attention mechanism, is the feature extraction network for the shallow features extracted by is the feature extraction network for the shallow features extracted by is the hyperparameter for controlling the penalty intensity, is the gradient penalty calculated by the gradient reversal layer GRL for , is the gradient operator, and the samples of the random interpolation domain are constructed as follows: Among them, is a random weight, and there is .

7. The method for adapting an electroencephalogram signal domain based on cross-attention according to claim 1, wherein When training the second classifier by freezing the network parameters of the discriminator and the global cross-attention aligner in step S105, the functional expression of the loss function used is: ， ， ， Among them, is the loss function used when training the second classifier with the network parameters of the freezing discriminator and the global cross-attention aligner. is the classification loss. is the adversarial loss. and are the numbers of electroencephalogram (EEG) signal data samples in the source domain and the target domain, respectively. is the number of EEG signal categories. is the label that the i-th EEG signal data sample belongs to the EEG signal category c. is the predicted value of the second classifier for the i-th EEG signal data sample belonging to the EEG signal category c. represents the EEG signal data sample from the source domain , is the discriminator. is the cross-attention mechanism. is the shallow features extracted by the feature extraction network for extracted.

8. An electroencephalogram signal domain adaptation system based on cross-attention, comprising a microprocessor and a memory connected to each other, characterized in that, The microprocessor is programmed or configured to execute the cross-attention based EEG signal domain adaptation method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program or instructions, characterized in that, The computer program or instruction is programmed or configured to execute the cross-attention based EEG signal domain adaptation method according to any one of claims 1 to 7 through a processor.

10. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instruction is programmed or configured to execute the cross-attention based EEG signal domain adaptation method according to any one of claims 1 to 7 through a processor.

Images