Brain-computer interface decoding model construction method and electroencephalogram signal recognition method

Abstract

The invention discloses a construction method of a brain-computer interface decoding model and an electroencephalogram signal recognition method, and belongs to the technical field of electroencephalogram detection.The construction method of the brain-computer interface decoding model comprises the steps that firstly, alignment operation is executed on electroencephalogram samples in an original calibration set of a target domain, and the accuracy of cross-user migration is improved; enhancing each electroencephalogram sample in the alignment calibration set to obtain a target data set, so that the generalization ability of the model can be improved; carrying out adversarial training on the initial brain-computer interface decoding model by utilizing adversarial samples generated by each electroencephalogram sample X in the target data set under a plurality of different random seeds to obtain an updated brain-computer interface decoding model corresponding to each random seed, and keeping high robustness of the model when facing the adversarial samples and noise data; and integrating the updated brain-computer interface decoding models corresponding to the random seeds to obtain a target brain-computer interface decoding model, thereby effectively improving the classification accuracy and antagonism robustness.

Description

Technical Field

[0001] The present invention belongs to the technical field of electroencephalogram (EEG) detection, and more specifically, relates to a method for constructing a brain-computer interface decoding model and an EEG signal recognition method. Background Art

[0002] Brain-Computer Interface (BCI) creates a direct communication pathway between the brain and external devices, enabling the user's cognitive and / or sensory-motor functions to be assisted, enhanced, or restored. BCI has been widely used in many fields such as neurorehabilitation, tactile exploration, robotic control, and language synthesis. BCI is mainly divided into three categories: non-invasive, partially invasive, and fully invasive. Among them, non-invasive BCI usually uses electroencephalogram (EEG) signals, which has become the most popular method due to its ease of use and low cost. Motor imagery (MI) is a classic paradigm in EEG-based BCI. In this process, the user activates different brain areas by imagining the movement of specific body parts (for example, the left hand, right hand, feet, or tongue), and these imagined movements can be converted into instructions to control external devices after decoding.

[0003] Although EEG-based BCI has advantages such as non-invasiveness and low cost, it faces at least the following three major challenges in practical application:

[0004] Accuracy: The acquisition process of EEG signals is time-consuming and laborious, so the amount of training data is usually small. In addition, EEG responses vary significantly between different users, making it difficult to directly apply a decoding model from one user to another user's data. This data scarcity and individual variability significantly reduce the accuracy of EEG decoding.

[0005] Adversarial vulnerability: BCI outputs are vulnerable to adversarial attacks. Previous studies have found that adversarial samples can significantly reduce the decoding accuracy of EEG-BCI, making it unreliable. In addition, subsequent studies have developed universal adversarial perturbations, but they have exposed serious security issues in BCI systems, making adversarial robustness a key consideration.

[0006] User privacy: EEG data contains a lot of sensitive personal information, such as credit card numbers, personal identification codes, known contacts and addresses, which may be inferred from EEG signals. In addition, studies have found that even resting EEG data can reveal the user's personality traits and cognitive abilities. Therefore, privacy protection has become an important issue that cannot be ignored in BCI systems.

[0007] A large number of studies have explored the above three challenges. For example, transfer learning improves BCI classification accuracy by using EEG data from other users; adversarial detection and robust training improve the robustness of BCI; privacy-preserving machine learning and data perturbations help protect the privacy of EEG data. However, solving a problem often introduces performance degradation in other aspects. For example, although cross-user transfer learning can improve the decoding accuracy of EEG models, it may lead to privacy leakage of source users; adversarial defense can improve the accuracy of the model under adversarial attacks, but it may reduce the classification accuracy of normal samples. In other words, there is currently no research that can solve all three challenges at the same time to achieve accurate, robust and privacy-preserving BCI decoding models. Summary of the invention

[0008] In response to the above defects or improvement needs of the prior art, the present invention provides a method for constructing a brain-computer interface decoding model and an electroencephalogram signal recognition method, the purpose of which is to solve the technical problem of how to construct an accurate, robust and privacy-protected BCI decoding model.

[0009] To achieve the above object, according to one aspect of the present invention, a method for constructing a brain-computer interface decoding model is provided, comprising:

[0010] S1: original calibration set for the target domain Perform alignment operations on the EEG samples in to obtain the aligned calibration set Among them, N T is the number of EEG samples in the target domain, is an EEG sample with c channels and t sampling points, {1,…,K} are the labels of EEG samples in the target domain, i=1,…,N T ;

[0011] S2: Align the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set

[0012] S3: The enhanced calibration set and the original calibration set Merge to get the target data set

[0013] S4: Using the target dataset under multiple different random seeds The adversarial samples generated by each EEG sample X in the training are used to perform adversarial training on the initial BCI decoding model, and an updated BCI decoding model corresponding to each random seed is obtained;

[0014] S5: Integrate the updated brain-computer interface decoding models corresponding to each of the random seeds to obtain a target brain-computer interface decoding model.

[0015] In one embodiment, S1 includes: using the formula The original calibration set for the target domain Perform an alignment operation on the EEG samples in to obtain the alignment calibration set Among them, the arithmetic mean of the spatial covariance matrix of all samples is

[0016] In one embodiment, S2 includes: using formula X" T,i =X′ T,i ·(1+β) or X" T,i =X′ T,i (1-β) Align the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set β is the scaling factor of the EEG samples.

[0017] In one of the embodiments, before S4, it also includes: in a centralized passive domain transfer learning scenario, performing intra-user alignment on the source domain data set of the source domain user, and training the initial brain-computer interface decoding model using the source domain data sets before and after the alignment.

[0018] In one of the embodiments, before S4, it also includes: in a federated passive domain transfer learning scenario, data between source domain users cannot be merged, the source domain data set of each user is aligned, and the initial brain-computer interface decoding model is obtained by training with the aligned source domain data set using a federated algorithm.

[0019] In one embodiment, the training optimization problem of updating the brain-computer interface decoding model in S4 is expressed as:

[0020]

[0021] in, X T is the center∈ and the radius is l ∞ Ball, C θ is the updated BCI decoding model with parameter θ, is the cross entropy loss function, X adv For adversarial samples.

[0022] In one embodiment, given an EEG sample X, an adversarial sample is generated using the following formula:

[0023] X adv,0 =X+ξ;

[0024]

[0025] Among them, X adv,i ∈[X adv,1 , ..., X adv,N-1 ], is the adversarial sample generated after iterating i steps of EEG sample X, ξ is the random noise uniformly sampled from the interval (-∈,∈), α≤∈ is the iteration step length, Proj X,∈ Guarantee X adv,i In the ∈ domain of X, and in accordance with l ∞ norm constraint, sign() is the sign function, is the gradient of the ith adversarial example, C θ (X adv,i-1 ) is the input X adv,i-1 Time Classifier C θ The output of y is the label of EEG sample X. For expectation.

[0026] In one of the embodiments, before S4, the step further includes: in a source data disturbance scenario: aligning a source domain data set of each user, and performing data disturbance on the source domain data set to obtain a source domain data set;

[0027] The S4 includes: using the disturbed source domain data set and the target data set The corresponding adversarial samples are used to perform adversarial training on the random initial brain-computer interface decoding model under multiple different random seeds to obtain the updated brain-computer interface decoding model corresponding to each random seed.

[0028] In one embodiment, the training optimization problem of updating the brain-computer interface decoding model in S4 is expressed as:

[0029]

[0030] in, X T is the center∈ and the radius is l ∞ Ball, C θ is the BCI decoding model with parameter θ, is the cross entropy loss function, is the generated target domain EEG adversarial sample, is the source domain EEG sample after perturbation, is the perturbed source domain EEG dataset, For expectation.

[0031] According to another aspect of the present invention, there is provided a device for constructing a brain-computer interface decoding model, comprising:

[0032] Alignment module, used to align the original calibration set of the target domain Perform alignment operations on the EEG samples in to obtain the aligned calibration set Among them, N T is the number of EEG samples in the target domain, is an EEG sample with c channels and t sampling points, y T,i ∈{1,…,K} is the label of the EEG sample in the target domain, i=1,…,N T ;

[0033] Enhancement module for aligning the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set

[0034] A merging module is used to merge the enhanced calibration set and the original calibration set Merge to get the target data set

[0035] A training module for utilizing the target dataset under multiple different random seeds The adversarial samples generated by each EEG sample X in the training are used to perform adversarial training on the initial BCI decoding model, and an updated BCI decoding model corresponding to each random seed is obtained;

[0036] An integration module is used to integrate the updated brain-computer interface decoding models corresponding to each of the random seeds to obtain a target brain-computer interface decoding model.

[0037] According to another aspect of the present invention, a method for recognizing EEG signals is provided, comprising: inputting the EEG signals of the current user of the brain-computer interface into the target brain-computer interface decoding model obtained by the above-mentioned method for constructing the brain-computer interface decoding model, and obtaining a recognition result corresponding to the EEG signals of the current user.

[0038] According to another aspect of the present invention, there is provided an electroencephalogram signal recognition device, comprising:

[0039] A device for constructing the above-mentioned brain-computer interface decoding model;

[0040] The recognition device is used to input the EEG signal of the current user of the brain-computer interface into the target brain-computer interface decoding model obtained by the above-mentioned brain-computer interface decoding model construction method to obtain the recognition result corresponding to the EEG signal of the current user.

[0041] According to another aspect of the present invention, there is provided an electronic device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned brain-computer interface decoding model construction method or EEG signal recognition method when executing the computer program.

[0042] According to another aspect of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for constructing the above-mentioned brain-computer interface decoding model or the method for recognizing electroencephalogram signals are implemented.

[0043] In general, the above technical solutions conceived by the present invention can achieve the following beneficial effects compared with the prior art:

[0044] (1) The present invention provides a method for constructing a brain-computer interface decoding model. First, an alignment operation is performed on the electroencephalogram samples in the original calibration set of the target domain, thereby reducing the signal distribution differences between different users and improving the accuracy of cross-user migration. Then, each electroencephalogram sample in the aligned calibration set is enhanced to obtain a target data set, thereby expanding the diversity of the data and thus improving the generalization ability of the model. The random initial electroencephalogram decoding model is adversarially trained using adversarial samples generated from the perturbed source domain data set and each electroencephalogram sample X in the target data set. Alternatively, the initial electroencephalogram decoding model (source domain model) obtained by training the source domain data set is adversarially trained using adversarial samples generated from each electroencephalogram sample X in the target data set to obtain an updated brain-computer interface decoding model corresponding to each random seed, wherein the model maintains high robustness when facing adversarial samples and noise data. The updated brain-computer interface decoding models corresponding to each random seed are integrated to obtain a target brain-computer interface decoding model. By training and integrating multiple randomly initialized neural networks, the classification accuracy and adversarial robustness are effectively improved. This application combines data alignment, data enhancement, adversarial training and ensemble learning to improve the classification accuracy and robustness of the target brain-computer interface decoding model, taking into account accuracy, adversarial robustness and privacy protection requirements.

[0045] (2) This scheme uses Euclidean alignment technology to spatially align EEG signals, reduce the signal distribution differences between different users, and improve the accuracy of cross-user migration.

[0046] (3) This scheme uses amplitude scaling data enhancement to enhance the target domain data. It is simple and easy to implement, does not change the relative size relationship between EEG channels, does not affect the task information carried by the EEG samples, and effectively expands the sample size.

[0047] (4) In the centralized passive domain transfer learning scenario, this scheme uses the model trained on the training set composed of multiple source domain EEG samples to initialize the target domain BCI decoding model. It not only effectively utilizes the EEG task knowledge carried by the source domain EEG samples, but also does not need to provide the target user with the source domain EEG samples, thus protecting the privacy of the source domain users; it also takes into account the privacy of the source domain users with respect to the target domain users, thus achieving the goal of protecting the privacy of the source domain users with respect to the target domain users while effectively utilizing the EEG task knowledge learned by the BCI decoding model on the source domain EEG samples to help improve the EEG decoding effect of the target domain users.

[0048] (5) In the federated passive domain transfer learning scenario, the EEG samples of each source domain cannot be grouped together to train the BCI decoding model. The EEG model of the source domain can only be trained on the EEG samples of each source domain in different time periods. The scheme takes into account the privacy of the source domain users with respect to the target domain users, as well as the privacy between the source domain users. It achieves the goal of learning EEG task knowledge from each EEG sample in the source domain while protecting the privacy of the source domain users with respect to the target domain users, as well as the privacy between the source domain users. The scheme also uses the knowledge of the source domain EEG samples learned by the model to help improve the EEG decoding effect of the target domain users.

[0049] (6) This scheme expresses the training optimization problem of updating the brain-computer interface decoding model as: The fragility of the brain-computer interface decoding model is taken into consideration, and the robustness of the brain-computer interface decoding model is enhanced.

[0050] (7) This scheme uses X adv,0 =X+ξ and Generate adversarial samples; Considering the time efficiency of generating adversarial samples and the diversity of generated samples, fast and diverse adversarial sample generation is achieved.

[0051] (8) In the scenario of source data disturbance, this scheme considers how to protect the user privacy information in the source domain dataset while sharing the source domain EEG data, and realizes the sharing of source domain EEG samples while protecting the user privacy information therein.

[0052] (9) This scheme expresses the training optimization problem of updating the brain-computer interface decoding model under the source data perturbation scenario as: Considering the use of EEG task knowledge in source domain samples, training time cost and improving target domain EEG decoding performance, it is possible to train a robust target domain brain-computer interface decoding model faster while utilizing source domain perturbation data. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 A flowchart of a method for constructing an EEG decoding model provided in Example 1 of the present invention;

[0054] Figure 2 A schematic diagram of the effects of data alignment, data enhancement, and adversarial training in the method for constructing an EEG decoding model provided in Example 1 of the present invention;

[0055] Figure 3 This is an application scenario diagram of the brain-computer interface system provided in Example 1 of the present invention;

[0056] Figure 4 A schematic diagram of an initial brain-computer interface decoding model in a centralized passive domain transfer learning scenario provided in Example 1 of the present invention;

[0057] Figure 5 A schematic diagram of an initial brain-computer interface decoding model in a federated passive domain transfer learning scenario provided in Example 1 of the present invention;

[0058] Figure 6 Schematic diagram of the initial brain-computer interface decoding model in the source domain data set disturbance scenario provided in Example 1 of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0060] Example 1

[0061] like Figure 1 and Figure 2 As shown, a method for constructing an EEG decoding model in this embodiment includes: S1-S5. Among them, Figure 3 The invention provides a brain-computer interface system for implementing EEG decoding using an algorithm for constructing an EEG decoding model.

[0062] S1: original calibration set for the target domain Perform alignment operations on the EEG samples in to obtain the aligned calibration set Among them, N T is the number of EEG samples in the target domain, is an EEG sample with c channels and t sampling points, y T,i ∈{1,…,K} is the label of the EEG sample in the target domain, i=1,…,N T S2: Align the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set S3: Enhanced calibration set and the original calibration set Merge to get the target data set S4: Using the adversarial samples generated by the perturbed source domain dataset and each EEG sample X in the target dataset, the random initial EEG decoding model is adversarially trained; or using the adversarial samples generated by each EEG sample X in the target dataset, the initial EEG decoding model (source domain model) trained by the source domain dataset is adversarially trained to obtain an updated EEG decoding model corresponding to each random seed. S5: The updated EEG decoding models corresponding to each random seed are fused to obtain a target EEG decoding model.

[0063] This embodiment aims to solve the challenges of current brain-computer interface decoding models in achieving accuracy, robustness and privacy protection at the same time, and proposes a method for constructing an EEG decoding model, which can be used to achieve accurate and robust BCI decoding while meeting the high accuracy, adversarial robustness and privacy protection requirements of BCI decoding.

[0064] Specifically, the method for constructing the EEG decoding model proposed in this embodiment includes the following steps:

[0065] 1. Data alignment: Spatial alignment of EEG signals to reduce the differences in signal distribution between different users and improve the accuracy of cross-user migration.

[0066] 2. Data augmentation: Perform data augmentation based on amplitude scaling on the target domain data to expand the diversity of the data and thus improve the generalization ability of the model.

[0067] 3. Adversarial training: Generate adversarial samples and perform training to ensure that the model remains highly robust when facing adversarial samples and noisy data.

[0068] 4. Ensemble learning: By training multiple randomly initialized neural networks and integrating their results, the classification accuracy and adversarial robustness are effectively improved.

[0069] Further, S1 includes: using the formula The original calibration set for the target domain Perform alignment operations on the EEG samples in to obtain the aligned calibration set Among them, the arithmetic mean of the spatial covariance matrix of all samples is

[0070] First, EA is performed on the samples in the calibration dataset of the target domain to reduce the data distribution difference. Data alignment makes the EEG data distribution more consistent. After EA processing, the EEG is whitened, that is, the average spatial covariance matrix becomes the unit matrix.

[0071] Further, S2 includes: using formula X" T,i =X′ T,i ·(1+β) or X" T,i =X′ T,i (1-β) will align the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set β is the scaling factor of the EEG samples.

[0072] The enhanced samples can be either X′=X·(1+β) or X′=X·(1-β). We perform two enhancement methods on each target domain sample. After data enhancement, the target calibration dataset of the target domain Contains original samples and enhanced samples.

[0073] This embodiment applies the construction method of the EEG decoding model to three privacy protection scenarios (centralized passive transfer learning, federated passive transfer learning, and source data perturbation) to achieve brain-computer interface decoding that is accurate, secure, robust, and privacy-protected.

[0074] Furthermore, before S4, it also includes: in a centralized passive domain transfer learning scenario, the source domain data sets of source domain users are first aligned within users and then aligned between users, and an initial EEG decoding model is trained using the aligned source domain data sets.

[0075] Among them, centralized passive domain transfer learning is shown in Figure 4 In this scenario, source users can share data with each other, but not directly with target users. The source user data can be combined and used to train the source domain model, and then only the source domain model is made public, thereby ensuring the privacy of the source user data. The method for constructing the EEG decoding model then optimizes the model with the target domain EEG samples on the given source domain model. The optimization steps are data alignment, data enhancement, adversarial training, and ensemble learning described above.

[0076] Furthermore, before S4, it also includes: in the federated passive domain transfer learning scenario, the data between source domain users cannot be merged, the source domain data set of each user is aligned, and the initial EEG decoding model is trained using the aligned source domain data set using a federated algorithm.

[0077] Among them, federated passive domain transfer learning can be found in Figure 5 In this scenario, the original data cannot be shared between source users or between source users and target users. A central server manages the global source domain model and distributes it to each source user for parameter update. The updated model is then aggregated to the global server, and only the global model can be shared. The construction method of the EEG decoding model then optimizes the model with the target domain EEG samples based on the given source domain model. The optimization steps are data alignment, data enhancement, adversarial training, and ensemble learning described above.

[0078] Furthermore, in order to improve the robustness of the model, this embodiment uses adversarial training of the EEG decoding model. Given a target domain calibration dataset after data enhancement and a neural network model (which can be an EEG model trained on the source domain dataset); the training optimization problem of updating the EEG decoding model in S4 is expressed as:

[0079]

[0080] in, X T is the center∈ and the radius is l ∞ Ball, C θ is the updated EEG decoding model with parameter θ, is the cross entropy loss function, X adv For adversarial samples.

[0081] Furthermore, given an EEG sample X, the following formula is used to generate adversarial samples:

[0082] X adv,0 =X+ξ;

[0083]

[0084] Among them, X adv,i ∈[X adv,1 , ..., X adv,N-1 ], is the adversarial sample generated after iterating i steps of EEG sample X, ξ is the random noise uniformly sampled from the interval (-∈,∈), α≤∈ is the iteration step length, Proj X,∈ Guarantee X adv,i In the ∈ domain of X, and in accordance with l ∞ norm constraint, sign() is the sign function, is the gradient of the ith adversarial example, Cθ (X adv,i-1 ) is the input X adv,i-1 Time Classifier C θ The output of y is the label of EEG sample X. For expectation.

[0085] Furthermore, before S4, it also includes: in the source data perturbation scenario: aligning the source domain data set of each user, and performing data perturbation on the source domain data set to obtain the source domain data set; S4 includes: using the perturbed source domain data set and the target data set For the random initial BCI decoding model, adversarial training is performed under multiple different random seeds to obtain the updated EEG decoding model corresponding to each random seed.

[0086] Among them, the source data disturbance scenario is shown in Figure 6 , by applying user-specific perturbations to the source domain dataset, the user's identity information cannot be learned, but the task-related information is still retained. The perturbed data can be used to assist the training of the target model without leaking the user's privacy. Given a source domain dataset N S is the number of samples in the target domain, For an EEG sample with c channels and t sampling points, y T,i ∈{1,…,K} is the source domain sample label, u i ∈{1,…,U} is the source domain user label. We S,i Adding user-based perturbations Generate a private and unlearnable EEG training dataset For each source domain EEG sample X S,i , This embodiment uses the synthetic noise method proposed in the literature [6] to calculate The construction method of the EEG decoding model then uses the perturbed source domain dataset and target domain data to optimize the randomly initialized EEG model. The steps are data alignment, data enhancement, adversarial training and ensemble learning described above.

[0087] Furthermore, in the source data perturbation scenario, a source domain dataset with privacy protection is provided. The EEG decoding model is randomly initialized; the training optimization problem of updating the EEG decoding model in S4 is expressed as:

[0088]

[0089] in, X Tis the center∈ and the radius is l ∞ Ball, C θ is the BCI decoding model with parameter θ, is the cross entropy loss function, is the generated target domain EEG adversarial sample, is the source domain EEG sample after perturbation, is the perturbed source domain EEG dataset, The model is optimized on the privacy-preserving samples in the source domain and the adversarial samples in the target domain.

[0090] Experimental verification shows that the method for constructing the EEG decoding model proposed in this embodiment is applied to three privacy-preserving transfer learning scenarios: centralized passive domain transfer learning, federated passive domain transfer learning, and source domain data set perturbation privacy protection. All of them train more accurate and robust EEG decoding models, as shown in Tables 1, 2, and 3. The EEG decoding model trained by the method for constructing the EEG decoding model proposed in this embodiment has higher accuracy on normal samples, adversarial samples, and noisy samples on the three EEG data sets of BNCI2014001, Weibo2014, and BNCI2014002. The decoding accuracy of the method for constructing the EEG decoding model on a variety of samples even exceeds that of the transfer learning algorithm without any perturbation, as shown in Table 4. Among them, Table 1 shows the classification accuracy (%) in centralized passive domain transfer learning. The "Avg." column represents the average of "Benign", "Adversarial", and "Noisy". “Average” represents the average results on the BNCI2014001, Weibo2014, and BNCI2014002 datasets. The highest accuracy in each column is marked in bold.

[0091] Table 1 Classification accuracy in centralized passive domain transfer learning

[0092]

[0093] Table 2 shows the classification accuracy (%) in the federated passive domain transfer learning scenario. The “Avg.” column represents the average of “Benign,” “Adversarial,” and “Noisy.” “Average” represents the average results on the BNCI2014001, Weibo2014, and BNCI2014002 datasets. The highest accuracy in each column is marked in bold.

[0094] Table 2 Classification accuracy in federated passive domain transfer learning scenario

[0095]

[0096] Table 3 shows the classification accuracy (%) in the source data perturbation scenario. The “Avg.” column represents the average of “Benign,” “Adversarial,” and “Noisy.” “Average” represents the average results on the BNCI2014001, Weibo2014, and BNCI2014002 datasets. The highest accuracy in each column is marked in bold.

[0097] Table 3 Classification accuracy in source data perturbation scenarios

[0098]

[0099] Table 4 Classification accuracy (%) without privacy protection. The “Average” column represents the average of “Benign”, “Adversarial” and “Noisy”. The highest accuracy in each column is marked in bold.

[0100] Table 4 Classification accuracy without privacy protection

[0101]

[0102] In summary, this embodiment proposes a method for constructing an EEG decoding model, which uses data alignment, data enhancement, adversarial training and ensemble learning to achieve more accurate and robust EEG decoding. This embodiment further applies the method for constructing an EEG decoding model to three privacy protection scenarios (centralized passive domain transfer learning, federated passive domain transfer learning, and source domain data set perturbation) to simultaneously achieve accurate decoding, adversarial robustness, and privacy-protected BCI. The experiment verified the effectiveness of the method for constructing the EEG decoding model, which is even better than the most advanced transfer learning method that does not consider privacy protection at all. This embodiment is the first to simultaneously deal with the three major challenges of data scarcity and individual differences, adversarial vulnerability, and user privacy in an EEG-based BCI, greatly improving the applicability of the EEG decoding algorithm in actual BCI systems.

[0103] Example 2

[0104] This embodiment provides a device for constructing a brain-computer interface decoding model, including: an alignment module, an enhancement module, a merging module, a training module and an integration module.

[0105] Among them, the alignment module is used to align the original calibration set of the target domain Perform alignment operations on the EEG samples in to obtain the aligned calibration set Among them, N T is the number of EEG samples in the target domain, is an EEG sample with c channels and t sampling points, y T,i∈{1,…,K} is the label of the EEG sample in the target domain, i=1,…,N T ; Enhanced module for aligning calibration sets Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set Merge module to combine enhanced calibration sets and the original calibration set Merge to get the target data set A training module is used to perform adversarial training on a random initial EEG decoding model using adversarial samples generated from the perturbed source domain dataset and each EEG sample X in the target dataset; or, using adversarial samples generated from each EEG sample X in the target dataset, perform adversarial training on an initial EEG decoding model (source domain model) obtained by training the source domain dataset to obtain an updated brain-computer interface decoding model corresponding to each random seed; an integration module is used to integrate the updated brain-computer interface decoding models corresponding to each random seed to obtain a target brain-computer interface decoding model.

[0106] Example 3

[0107] This embodiment provides a method for recognizing EEG signals, including: inputting the EEG signals of the current user of the brain-computer interface into the target brain-computer interface decoding model obtained by the method for constructing the brain-computer interface decoding model in Embodiment 1, and obtaining the recognition result corresponding to the EEG signals of the current user. Through the collaborative work of the method for constructing the EEG decoding model and the EEG signal recognition, the brain-computer interface can simultaneously achieve accurate decoding, robust security, and privacy protection.

[0108] Example 4

[0109] The present embodiment provides an EEG signal recognition device, comprising: a device for constructing the brain-computer interface decoding model in Example 1 and a recognition device, which is used to input the EEG signal of the current user of the brain-computer interface into the target brain-computer interface decoding model obtained by the method for constructing the brain-computer interface decoding model in Example 1, and obtain a recognition result corresponding to the EEG signal of the current user.

[0110] Example 5

[0111] This embodiment provides an electronic device including a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the steps of the method for constructing a brain-computer interface decoding model or the method for recognizing electroencephalogram signals in Embodiment 1 are implemented.

[0112] Example 6

[0113] This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for constructing a brain-computer interface decoding model or the method for recognizing electroencephalogram signals in Embodiment 1 are implemented.

[0114] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for constructing a brain-computer interface decoding model, characterized in that: include: S1: original calibration set for the target domain Perform alignment operations on the EEG samples in to obtain the alignment calibration set Among them, N T is the number of EEG samples in the target domain, is an EEG sample with c channels and t sampling points, y T,i ∈{1,…,K} is the label of the EEG sample in the target domain, i=1,…,N T ; S2: Align the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set S3: The enhanced calibration set and the original calibration set Merge to get the target data set S4: Using the target dataset under multiple different random seeds The adversarial samples generated by each EEG sample X in the training are used to perform adversarial training on the initial BCI decoding model, and an updated BCI decoding model corresponding to each random seed is obtained; S5: Integrate the updated brain-computer interface decoding models corresponding to each of the random seeds to obtain a target brain-computer interface decoding model.

2. The method for constructing a brain-computer interface decoding model according to claim 1, characterized in that: The S1 includes: Using the formula The original calibration set for the target domain Perform an alignment operation on the EEG samples in to obtain the alignment calibration set Among them, the arithmetic mean of the spatial covariance matrix of all samples is 3. The method for constructing a brain-computer interface decoding model according to claim 1, characterized in that: The S2 includes: using formula X" T,i =X′ T,i ·(1+β) or X" T,i =X′ T,i (1-β) Align the calibration set Each EEG sample X′ in T,i Enhanced, to obtain enhanced EEG sample X" T,i , the corresponding enhanced calibration set β is the scaling factor of the EEG samples.

4. The method for constructing a brain-computer interface decoding model according to claim 1, characterized in that: Before S4, it also includes: in a centralized passive domain transfer learning scenario, performing intra-user alignment on the source domain data set of the source domain user, and training the initial brain-computer interface decoding model using the source domain data sets before and after the alignment.

5. The method for constructing a brain-computer interface decoding model according to claim 1, characterized in that: Prior to S4, this also included: In the federated passive domain transfer learning scenario, the data between source domain users cannot be merged. The source domain data set of each user is aligned, and the initial brain-computer interface decoding model is obtained by training the aligned source domain data set using a federated algorithm.

6. The method for constructing a brain-computer interface decoding model as claimed in claim 4 or 5, characterized in that: The training optimization problem of updating the brain-computer interface decoding model in S4 is expressed as: in, X T is the center∈ and the radius is l ∞ Ball, C θ is the updated BCI decoding model with parameter θ, is the cross entropy loss function, X adv For adversarial samples.

7. The method for constructing a brain-computer interface decoding model as claimed in claim 6, characterized in that: Given an EEG sample X, use the following formula to generate adversarial samples: X adv,0 =X+ξ; Among them, X adv,i ∈[X adv,1 , ..., X adv,N-1 ], is the adversarial sample generated after iterating i steps of EEG sample X, ξ is the random noise uniformly sampled from the interval (-∈,∈), α≤∈ is the iteration step length, Proj X,∈ Guarantee X adv,i In the ∈ domain of X, and in accordance with l ∞ norm constraint, sign() is the sign function, is the gradient of the i-1th adversarial example, C θ (X adv,i-1 ) is the input X adv,i-1 Time Classifier C θ The output of y is the label of EEG sample X. For expectation.

8. The method for constructing a brain-computer interface decoding model according to claim 1, characterized in that: Before S4, it also includes: in the source data perturbation scenario: aligning the source domain data set of each user, and performing data perturbation on the source domain data set to obtain the source domain data set; The S4 includes: using the disturbed source domain data set and the target data set The corresponding adversarial samples are used to perform adversarial training on the random initial brain-computer interface decoding model under multiple different random seeds to obtain the updated brain-computer interface decoding model corresponding to each random seed.

9. The method for constructing a brain-computer interface decoding model as claimed in claim 8, characterized in that: The training optimization problem of updating the brain-computer interface decoding model in S4 is expressed as: in, X T is the center∈ and the radius is l ∞ Ball, C θ is the BCI decoding model with parameter θ, is the cross entropy loss function, is the generated target domain EEG adversarial sample, is the source domain EEG sample after perturbation, is the perturbed source domain EEG dataset, For expectation.

10. A method for recognizing an electroencephalogram signal, characterized in that: include: The EEG signal of the current user of the brain-computer interface is input into the target brain-computer interface decoding model obtained by the method for constructing the brain-computer interface decoding model described in any one of claims 1-6 to obtain a recognition result corresponding to the EEG signal of the current user.

Images