EEG signal cross-device decoding method with multi-source domain sample reweighting

Abstract

The application discloses a multi-source domain sample reweighting EEG signal cross-device decoding method, comprising the following steps: obtaining a multi-source domain sample data set by collecting MI EEG signals of multiple subjects and a target domain sample data set by collecting MI EEG signals of a target subject; training a preset SRENet network by using the two data sets to obtain a trained target SRENet network; obtaining a decoding result of a to-be-tested sample by using the target SRENet network; wherein the preset SRENet network comprises a feature extractor, a sample reweighting classifier and a conditional reweighting discriminator; in the network training process, the sample reweighting classifier and the conditional reweighting discriminator are used to measure the easy migration of the features extracted by the feature extractor, and a source domain sample which is more easily migrated to the target domain is given a higher training weight. The application can improve the decoding performance of the cross-device MI EEG signal.

Description

Technical Field

[0001] This invention belongs to the field of electroencephalogram (EEG) signal processing, specifically relating to a cross-device decoding method for multi-source domain sample reweighting of EEG signals. Background Technology

[0002] Brain-computer interfaces (BCIs) establish a direct communication method between the human brain and a computer by recording and decoding brain signals. Current BCI paradigms mainly include: motor imagery, steady-state visual evoked potentials (SVPs), P300, slow cortical potentials (SVPs), and mental state monitoring. Among these, the motor imagery (MI) paradigm, as the name suggests, involves the activation of specific brain regions even when a person imagines their limbs (or muscles) moving without actual physical movement. By analyzing EEG signals and detecting the activation effects of different brain regions, the user's intentions can be determined, thereby enabling direct communication and control between the human brain and external devices. Commonly used areas for motor imagery include the left hand, right hand, both feet, and tongue.

[0003] Motor imagery BCI based on electroencephalogram (EEG) has promising applications due to its non-invasive and stimulation-free nature. In laboratories, researchers typically use expensive specialized equipment to record MI EEG signals, resulting in a vast amount of MI EEG data collected in laboratories worldwide. However, in practical applications, the high cost of specialized laboratory equipment often forces the use of cheaper equipment for data acquisition. This leads to lower channel counts and lower EEG signal quality compared to laboratory data, resulting in discrepancies. Furthermore, the limited number of training samples in practical tests, constrained by experimental costs, hinders the learning process and wastes valuable laboratory data. Cross-device transfer learning promises to address this issue. Effective cross-device transfer learning allows for the full utilization of existing laboratory datasets to train MI EEG decoding algorithms on real-world devices, thereby significantly improving the performance of MI EEG signal decoding algorithms in practical applications.

[0004] Currently, there are several transfer learning methods for the EEG domain. For example, Li et al. proposed a bidirectional LSTM (BiLSTM) network, which combines the domain discriminator and the classifier to reduce the domain offset between training and test data; Hang et al. proposed a deep domain adaptive network (DDAN) for in-subject EEG recognition, etc.

[0005] Cross-device MI EEG decoding tasks typically involve multiple source domains, with only some of these domains readily transferable. However, most existing MI EEG decoding methods fail to effectively utilize data from these multi-source domains that are beneficial for target domain classification during training. Furthermore, EEG signals are multi-channel, one-dimensional, non-stationary signals, requiring feature extractors in decoding methods to possess sufficient robustness and adaptability to different domain data, a requirement that current methods cannot meet.

[0006] Therefore, how to propose a new transfer learning method to further improve the decoding performance of MI EEG signals across devices is a hot research topic worthy of study in this field. Summary of the Invention

[0007] The purpose of this invention is to provide a cross-device decoding method for EEG signals with multi-source domain sample reweighting, so as to improve the decoding performance of MI EEG signals across devices. The specific technical solution is as follows:

[0008] A multi-source domain sample dataset is obtained by collecting MI EEG signals from multiple subjects, and a target domain sample dataset is obtained by collecting MI EEG signals from a target subject; wherein, the multi-source domain sample dataset includes multiple source domain samples and corresponding classification labels; the target domain sample dataset includes multiple target domain samples;

[0009] The pre-defined SRENet network is trained using the multi-source domain sample dataset and the target domain sample dataset to obtain the trained target SRENet network. The pre-defined SRENet network includes a feature extractor, a sample reweighting classifier, and a conditional reweighting discriminator. During network training, the transferability of the features extracted by the feature extractor is measured using the sample reweighting classifier and the conditional reweighting discriminator, and source domain samples that are more easily transferred to the target domain are assigned higher training weights.

[0010] The decoding result of the test sample is obtained using the target SRENet network; wherein the test sample is obtained by collecting the MI EEG signal of the target subject.

[0011] The beneficial effects of this invention are:

[0012] This invention provides a method for cross-device decoding of EEG signals using multi-source domain sample reweighting. It utilizes labeled multi-source domain sample datasets and unlabeled target domain sample datasets obtained from the subject's MI EEG signals to train a pre-constructed SRENet network, resulting in a trained target SRENet network. The target SRENet network is then used to obtain the decoding results of the test samples. The SRENet network in this invention includes a feature extractor, a sample reweighting classifier, and a conditional reweighting discriminator. It establishes a MI EEG transfer learning network between the multi-source domain and a target domain, and is also a sample reweighted EEG decoding network. A sample reweighting strategy is employed to improve cross-device MI EEG decoding performance. During network training, the sample reweighting classifier and the conditional reweighting discriminator measure the transferability of features extracted by the feature extractor. Source domain samples that are more easily transferred to the target domain are assigned higher training weights. Therefore, it can effectively utilize data from the multi-source domain data that is beneficial for target domain classification for effective training, further improving the cross-device MI EEG signal decoding performance through a novel transfer learning method. Furthermore, the weight allocation of samples by the sample reweighting classifier and the conditional reweighting discriminator during training plays a constraining role on the feature extractor during backpropagation, enabling the feature extractor to extract more general features for both the source and target domains. This improves the robustness and adaptability to data from different domains.

[0013] Furthermore, in the network training process, this embodiment of the invention automatically reweights all source domain data based on the feature distance and classification confidence between the source and target domains, thereby accurately predicting the labels of the target domain data. It can effectively utilize MI EEG data from multiple source domains to train cross-device decoding algorithms, which has significant application value for deploying decoding algorithms on new devices. Attached Figure Description

[0014] Figure 1 This is a flowchart illustrating a method for cross-device decoding of EEG signals with multi-source domain sample reweighting provided in an embodiment of the present invention.

[0015] Figure 2 This is a schematic diagram of the structure of the feature extractor in the preset SRENet network according to an embodiment of the present invention;

[0016] Figure 3 This is a schematic diagram of the structure of a one-dimensional multichannel convolutional layer used by the feature extractor in the preset SRENet network of this invention embodiment;

[0017] Figure 4 This is a schematic diagram of the sample reweighting classifier in the preset SRENet network of this invention.

[0018] Figure 5 This is a schematic diagram of the structure of a conditional reweighting discriminator in a pre-defined SRENet network according to an embodiment of the present invention;

[0019] Figure 6 This is a schematic diagram of a pre-defined SRENet network structure according to an embodiment of the present invention;

[0020] Figure 7 This is a comparison chart of the average accuracy of each comparison method on three transmission tasks in the experiments of this embodiment of the invention;

[0021] Figure 8(a) shows the feature visualization results marked with domain labels when the network does not contain a sample reweighting classifier and a conditional reweighting discriminator in the experiment of the embodiment of the present invention.

[0022] Figure 8(b) shows the feature visualization results marked with classification labels when the network does not contain a sample reweighting classifier and a conditional reweighting discriminator in the experiment of the embodiment of the present invention;

[0023] Figure 8(c) shows the feature visualization results marked with domain labels when the network contains a sample reweighted classifier and a conditional reweighted discriminator in the experiment of the embodiment of the present invention.

[0024] Figure 8(d) shows the feature visualization results marked with classification labels when the network contains a sample reweighted classifier and a conditional reweighted discriminator in the experiment of the embodiment of the present invention;

[0025] Figure 9 This is a diagram showing the relationship between the feature distribution and classification ability of each source domain in the experiments of this embodiment of the invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] To further improve the decoding performance of cross-device MI EEG signals, this invention provides a cross-device decoding method for EEG signals with multi-source domain sample reweighting.

[0028] It should be noted that the execution entity of the multi-source domain sample reweighted EEG signal cross-device decoding method provided in this embodiment of the invention can be a multi-source domain sample reweighted EEG signal cross-device decoding device, which can run in an electronic device. This electronic device can be a server or a terminal device, but is not limited to these.

[0029] like Figure 1 As shown in the figure, the cross-device decoding method for multi-source domain sample reweighting of EEG signals provided by the embodiments of the present invention may include the following steps:

[0030] S1, acquire the multi-source domain sample dataset obtained by collecting MI EEG signals from multiple subjects, and the target domain sample dataset obtained by collecting MI EEG signals from the target subject.

[0031] The multi-source domain sample dataset includes multiple source domain samples and corresponding classification labels; the target domain sample dataset includes multiple target domain samples. In this embodiment, the MIEEEG signal acquisition devices for the source domain and the target domain are different. The source domain can correspond to expensive and professional laboratory equipment for acquiring MIEEEG signals, while the target domain can correspond to lower-cost practical application equipment for acquiring MIEEEG signals. Of course, this is just an example; it is understood that the MIEEEG signal acquisition device and the subject are exactly the same, representing the same data domain. Typically, the multi-source domain sample dataset includes source domain samples corresponding to multiple subjects, hence it is multi-source domain, while the target domain sample dataset may only include target domain samples corresponding to one target subject.

[0032] To facilitate understanding of the embodiments of the present invention, the acquisition experiments of MI EEG signals in the multi-source domain sample dataset and the target domain sample dataset will be briefly described first.

[0033] For the experiment using a multi-source domain sample dataset, several healthy subjects who met the experimental requirements were first selected. Subjects were provided with a continuously switching sequence of images on a display screen. They observed the targets in each image and imagined their motion, while MI EEG signals were acquired using a data acquisition device. An example illustration is given using the motion imagination task of clenching the left and right hands. Specifically:

[0034] 1) When the first target appears on the screen or when the target appears in the first position, the subject is asked to imagine clenching his left hand.

[0035] 2) When a second target appears on the screen or when the target appears in a second position, the subject is asked to imagine clenching his right hand.

[0036] In this context, the target appears in the form of an image. Different motion images can be distinguished by the content of the corresponding image, for example, the first target could be an animal image, the second target could be a plant image, etc. Alternatively, different motion images can be distinguished by the position in which the target appears, for example, the first position could be the left side of the screen, the second position could be the right side of the screen, and so on. Of course, the ways to distinguish different motion images are not limited to the above examples.

[0037] In the experiment, the sampling frequency and the number of channels for acquiring EEG data can be reasonably selected as needed, and no specific limitations are made here.

[0038] It is understandable that, regarding the motor imagery in 1) and 2) above, since the subjects are informed of the task, the classification of the collected MI EEG signals as either imagining clenching the left hand or imagining clenching the right hand is known. That is, the collected MI EEG signals are labeled, and the labels can be used as classification tags for the corresponding MI EEG signals.

[0039] Meanwhile, to remove noise and other influences, the acquired MI EEG signals are usually preprocessed, including noise reduction, such as power frequency filtering and 1-49Hz bandpass filtering. The specific preprocessing steps can be set appropriately as needed, and will not be described in detail here.

[0040] For the experiment on the multi-source domain sample dataset, each preprocessed MI EEG signal is treated as a source domain sample, and all source domain samples together with their corresponding classification labels constitute the multi-source domain sample dataset.

[0041] The experiment for the target domain sample dataset was similar to that for the multi-source domain sample dataset. The difference was that the target subjects were not asked to imagine whether to clench their left or right hand. Therefore, the collected MI EEG signals were unlabeled and their classification results were unknown.

[0042] For ease of understanding in the following text, the data formats of multi-source domain sample datasets and target domain sample datasets are defined below.

[0043] A multi-source domain sample dataset is defined as:

[0044]

[0045] The target domain sample dataset is defined as follows:

[0046]

[0047] Where s represents the source domain and t represents the target domain; This represents the i-th source domain sample; express Category tags; n s This indicates the number of samples in a multi-source domain sample dataset; Represents the j-th target domain sample; n t This indicates the number of samples in the target domain sample dataset; C represents the number of channels; T represents the number of time points.

[0048] In addition, each source domain sample and target domain sample also carries the domain label y of that sample. dThis indicates whether the sample originates from the target domain. For example, 0 indicates that the sample does not originate from the target domain, while 1 indicates that the sample originates from the target domain.

[0049] S2, using multi-source domain sample datasets and target domain sample datasets to train the preset SRENet network, and obtain the trained target SRENet network.

[0050] The pre-defined SRENet network includes a feature extractor, a sample reweighting classifier, and a conditional reweighting discriminator. During network training, the sample reweighting classifier and the conditional reweighting discriminator measure the transferability of the features extracted by the feature extractor, and assign higher training weights to source domain samples that are more easily transferred to the target domain.

[0051] To facilitate understanding of the target SRENet network in this embodiment of the invention, the structure of the preset SRENet network will first be described.

[0052] (1) Feature Extractor

[0053] The feature extractor, denoted by F, is used to extract features from the source domain samples and the target domain samples in each mini-batch of input samples and output the corresponding features.

[0054] Each mini-batch consists of multiple source domain samples and multiple target domain samples. To facilitate the same training process, the number of samples in each mini-batch, as well as the ratio of source domain samples to target domain samples, can be fixed. For example, the number of samples in each mini-batch can be 64, which includes 32 source domain samples and 32 target domain samples.

[0055] The feature extractor can be constructed using any existing feature extraction network structure, and no specific limitations are imposed here.

[0056] For example, in one alternative implementation, the feature extractor may include:

[0057] The layers are connected in sequence: the first convolutional layer, the first batch normalization layer, the second convolutional layer, the second batch normalization layer, the activation function layer, the average pooling layer, the random discarding layer, and the data flattening layer.

[0058] Please see Figure 2 , Figure 2 This is a schematic diagram of the feature extractor structure in the preset SRENet network of this invention.

[0059] The feature extractor is based on a simple two-layer convolutional network, namely a first convolutional layer and a second convolutional layer, to extract features from MIEEG data. Both the first and second convolutional layers are one-dimensional multi-channel convolutional layers used to extract the information contained in the MIEEG signal into the feature space. The feature extractor uses two one-dimensional multi-channel convolutional layers to extract the information contained in the MIEEG signal into the feature space. For the specific structure of the one-dimensional multi-channel convolutional layer, please refer to [link to documentation]. Figure 3 In this one-dimensional multi-channel convolutional layer, each channel of the convolutional kernel converts the input information into features, and the features from all channels are summed to form a single output channel feature. This one-dimensional multi-channel convolutional layer simultaneously extracts spatiotemporal features, avoiding overfitting to a specific input channel. The hyperparameters F1 and K control the number and length of the convolutional kernels in each layer.

[0060] After the first and second convolutional layers, first and second batch normalization layers are applied respectively to provide regularization. Following the second batch normalization layer, an activation function layer is used to provide a non-linear transformation; the activation function can be ReLU, etc. Then, an average pooling layer is used to reduce the dimensionality of the features, where the kernel length is controlled by K. Finally, a random dropout layer is used to reduce overfitting, and a data flattening layer is used to flatten the data.

[0061] Data samples in the source and target domains and After being input into the proposed feature extractor F, features from the source and target domains can be obtained. and

[0062] (2) Sample reweighting classifier

[0063] The sample reweighting classifier is denoted by C.

[0064] Please see Figure 3 In one optional implementation, the sample reweighting classifier may include:

[0065] The weight calculation module is used during network training to determine the distance between the features of each source domain sample and the center of the target domain feature space, as well as the dispersion of the features of the source domain samples and the features of the target domain samples, based on the features of the source domain samples and the target domain samples output by the feature extractor. The module then uses these distances and dispersion to calculate the weights of the corresponding source domain samples.

[0066] The classification module is used to classify the features of each source domain sample and each target domain sample during network training to obtain the corresponding classification results;

[0067] The source domain sample error calculation module is used to obtain the classification error of each source domain sample based on the classification results and corresponding classification labels of each source domain sample during network training.

[0068] The sample reweighted classifier loss function calculation module is used to weight the classification error of each source domain sample during network training, and obtain the value of the sample reweighted classifier loss function by averaging the weighted results of all source domain samples.

[0069] The sample reweighting classifier (SRC) proposed in this embodiment of the invention aims to increase the training weights of samples in the source domain feature space that are closer to the target domain, and to reduce the influence of source domain samples that are too different from the target domain in the feature space.

[0070] Specifically, for the weight calculation module, based on the features of the source and target domain samples output by the feature extractor, the distance between the features of each source domain sample and the center of the target domain feature space, as well as the dispersion of the features of the source and target domain samples, are determined in the corresponding mini-batch samples. The weights of the corresponding source domain samples are then calculated using these distances and dispersion values, including:

[0071] A1, based on the features of all target domain samples output by the feature extractor. Calculate the center of the target domain feature space in the corresponding mini-batch samples.

[0072] Where, d t This represents domain t, i.e., domain t; The features of the j-th sample in domain t; This indicates that sample x is the j-th sample in domain t, i.e. In this context, t represents the sample originating from the domain t; E is the mathematical expectation, which is the center of the feature space formed by all the features of this batch of data, serving as the center of the feature space of the domain t. m stands for measure.

[0073] This step calculates the center of the target domain feature space in a minibatch to represent the location of the target domain. Specifically, it is achieved by calculating the expected value E. For details, please refer to the existing technology. It will not be explained in detail here.

[0074] A2, for the features of each source domain sample output by the feature extractor. Calculate the features of the source domain sample With the center of the target domain feature space distance

[0075] Specifically, the features f of the source domain samples i s With the center of the target domain feature space The distance between them is defined as in It can be Euclidean distance, etc.

[0076] A3. Using the distances between the features of all source domain samples and the center of the target domain feature space, calculate the dispersion of the features of the source domain samples and the features of the target domain samples in the corresponding mini-batch.

[0077]

[0078] Where, d t This refers to domain s; This indicates that sample x is the i-th sample in the domain s.

[0079] The embodiments of the present invention use average deviation This is used to express the degree of dispersion between multi-source domain features and target domain features in a minibatch, and is used to calculate the reweighting decay rate of source domain samples.

[0080] The calculation of σ is achieved by obtaining the expected value E. For details, please refer to the existing technology for understanding, which will not be explained in detail here.

[0081] A4, for each source domain sample, use the distance of that source domain sample. Given the dispersion degree σ, the weights of the source domain samples are calculated using a preset weight calculation formula.

[0082] In one optional implementation, a Gaussian function is used as the reweighting function. The preset weight calculation formula is:

[0083]

[0084] in, This represents the weight of the i-th source domain sample.

[0085] According to the weight calculation formula above, for a source domain sample... If its characteristic f i s The center of the target domain feature space calculated from the target domain samples in the same mini-batch. Distance between The smaller the value, the higher the corresponding weight. The larger the weight, the greater the weight; conversely, the smaller the weight, the greater the weight. The smaller.

[0086] As can be seen, the sample reweighting classifier calculates the classification error of the source domain samples and reweights them according to the inter-domain distance between the features of each source domain sample and the target domain. This results in source domain data that are closer to the target domain data receiving higher weights. In other words, assigning corresponding weights to different source domain samples can increase the training weights of samples in the source domain feature space that are closer to the target domain (with lower inter-domain distance) and reduce the influence of source domain samples that are too different from the target domain in the feature space.

[0087] Specifically, the classification module categorizes the features of each input sample, yielding a corresponding classification result. Taking the motion imagery experiment in S1 as an example, the classification result indicates whether the subject's MI EEG signal corresponds to imagining clenching their left hand or their right hand. In essence, the classification module is a classifier, and the classification result is the classifier's output.

[0088] Therefore, the classification module is used to classify the source domain samples and target domain samples during network training to obtain the corresponding classification results. For ease of understanding, the classification results of the source domain samples are represented as follows: The classification result of the target domain sample is represented as follows: It is also used in actual testing processes to classify the samples to be tested and obtain the corresponding classification results.

[0089] The classification module can be implemented using any existing classification network, such as a fully connected (FC) layer.

[0090] Regarding the source domain sample error calculation module, it can be understood that during network training, after the forward propagation of the classification module, the source domain sample error calculation module can use the classification label of each source domain sample and the classification result of the classification module to calculate the classification error of each source domain sample.

[0091] The classification error of each source domain sample is obtained based on the classification results and corresponding classification labels. The calculation formula used includes:

[0092]

[0093] Among them, the source label is obtained by using cross-entropy. Compared with classification results The classification error between them is the classifier loss. i (θ F ,θ C ) represents the classification error of the i-th source domain sample; θ F θ represents the parameters of the feature extractor. C These represent the parameters of the sample reweighting classifier; Represents the classification label of the i-th source domain sample; represents the classification result of the i-th source domain sample; c represents the c-th data category in a classification result; M represents the total number of data categories in the classification result.

[0094] Understandably, a classification result is represented as a vector, where each value represents the confidence level of each data category. Taking the motion imagery experiment in S1 as an example, the classification result has two values: the confidence level of imagining clenching the left hand and the confidence level of imagining clenching the right hand. Therefore, in this case, there are two data categories, i.e., M is 2.

[0095] For the loss function calculation module of the sample reweighted classifier, during the network training process, W is used. i s E represents i (θ F ,θ C The weights of the samples are multiplied by the weights of the samples, and the product of the two is taken as the reweighting error of the corresponding source domain sample. Then, the average reweighting error of all source domain samples is calculated as the loss of the SRC. That is, the expression of the sample reweighting classifier loss function is as follows:

[0096]

[0097] Among them, L RC (θ F θ C θ represents the loss function of the sample reweighted classifier, also known as the Classifier Reweighted Loss; F θ represents the parameters of the feature extractor. C These represent the parameters of the sample reweighting classifier; E represents the weight of the i-th source domain sample; i Let represent the classification error of the i-th source domain sample; N represents the total number of source domain samples.

[0098] Understandably, the above formula can be used to calculate the value of the loss function of the sample reweighted classifier.

[0099] (3) Conditional reweighting discriminant

[0100] The conditionally reweighted discriminator, denoted by D, is used only during the training process.

[0101] Please see Figure 4 In one optional implementation, the conditional reweighting discriminator may include:

[0102] The conditional feature calculation module is used during network training to take the classification result of the samples in the source domain and the target domain output by the classification module as the condition of the feature extractor output by the feature extractor for each domain. The conditional features of each domain are combined with the corresponding conditions to obtain the conditional features of each sample in the domain.

[0103] Gradient inversion layers are used during network training to leave the forward propagation process unchanged, while inverting the gradient during the back propagation process by multiplying by a negative scalar.

[0104] The discriminant module is used to determine the domain category of each source domain sample and each target domain sample based on the corresponding conditional features during network training, and obtain the corresponding domain discrimination results to confuse the source domain and target domain during backpropagation, making the features of the two domains more similar. The conditional reweighted discriminator loss function calculation module is used to calculate the value of the conditional reweighted discriminator loss function based on the domain discrimination results of each source domain sample and each target domain sample output by the discriminant module, as well as the conditional features of each sample in each domain, during network training.

[0105] This invention employs a conditional reweighted discriminator to calibrate the feature distribution between the source and target domains. Specifically, a conditional reweighted discriminator loss function is designed to increase the weights of transfer-easily transferable samples during transfer learning. This allows data samples with higher classification confidence to receive higher weights during domain adaptation. Specifically, for the conditional feature calculation module, features of each source and target domain sample are obtained from the feature extractor, i.e., each f... i s and On the other hand, the classification module of the sample reweighting classifier obtains the classification results of the features of each source domain sample and target domain sample, that is, each and Will As f i s The conditions will As The conditions are determined, and the conditions and features of each sample are combined to obtain the corresponding conditional features.

[0106] In one optional implementation, the conditional features of each sample in the domain are obtained by combining the features and conditions of the domain, including:

[0107] For each feature, perform a multilinear mapping operation between the condition of that feature and the feature itself to obtain a conditional feature matrix for that feature:

[0108]

[0109] Where h represents the conditional feature matrix of a sample; p = [p1, p2, ..., p...] M [] represents the classification result of the sample, where each element in p represents the confidence level of the corresponding data category, and M represents the total number of data categories in the classification result; f = [f1, f2, ..., f d ] represents the feature of the sample; d represents the dimension of the feature; This indicates a multilinear mapping operation.

[0110] Taking the motion imagery experiment in S1 as an example, the classification result has two values: the confidence level of imagining clenching the left hand and the confidence level of imagining clenching the right hand. Therefore, for any sample obtained by the classification module, the classification result p is p = [p1, p2], where p1 can be the confidence level of imagining clenching the left hand and p2 can be the confidence level of imagining clenching the right hand.

[0111] The training process for each mini-batch sample is divided into forward propagation and backward propagation. For the gradient inversion layer, no processing is done during the forward propagation process, but the gradient is inverted by multiplying by a negative scalar during the backward propagation process.

[0112] The discrimination module outputs the domain discrimination result, which indicates whether the sample comes from the source domain or the target domain. The working process of the discrimination module can include the following steps:

[0113] ① Training the discrimination module: Use the features extracted by the feature extractor to train the discrimination module, so that it can determine whether the sample comes from the source domain or the target domain;

[0114] ② Overall forward propagation: The conditional feature matrix h of each sample is used as a conditional feature and directly input into the discrimination module. The discrimination module determines whether the sample comes from the source domain or the target domain based on the input h, and obtains the domain discrimination result.

[0115] ③ Use the domain discrimination results for backpropagation to supervise the training of the feature extractor: fix the parameters of the discrimination module, and reverse the gradient of the GRL layer during backpropagation to supervise the training of the feature extractor, so that the features extracted by the feature extractor cannot be judged by the discrimination module to come from the source domain or the target domain.

[0116] The role of the discriminator module is to use the sample with the highest conditional weight to supervise the feature extractor during backpropagation to confuse the source and target domains. The purpose is to make the features of the source and target domains more similar, which is beneficial to the final sample classifier, i.e., the classification module in the sample reweighted classifier.

[0117] By iterating the above steps repeatedly during training, the source and target domain features can be made more similar. The discrimination module can be implemented using any existing classification network, such as a fully connected (FC) layer.

[0118] In one optional implementation, the expression for the conditional reweighted discriminator loss function includes:

[0119]

[0120] in, This represents the classification result of the i-th source domain sample; Let represent the conditional feature matrix of the i-th source domain sample; This represents the classification result of the j-th target domain sample; Let represent the conditional feature matrix of the j-th target domain sample; D(h) represents the domain discrimination result of the discrimination module on the conditional feature matrix of a sample; w(H(p)) represents the sample weight; H(p) represents the entropy criterion.

[0121] in,

[0122] w(H(p))=1+e -H(p)

[0123]

[0124] p n This represents the classification result of a classifier, where M represents the total number of data categories in the classification result, and p... n The discriminant module's decision result; H(p) is a metric used to quantify the uncertainty of the classifier's predictions. w(H(p)) reweights each sample to increase the weights for lower uncertainty in the classifier, allowing data samples with higher classification confidence to receive higher weights during domain adaptation. This entropy-based loss function encourages highly deterministic predictions. Therefore, the conditionally reweighted discriminator helps improve classification performance for unlabeled data samples across devices.

[0125] In this embodiment of the invention, a loss function L is designed. D (θ F ,θ D To increase the weight of transfer-easily transferable samples during transfer learning, specifically: L D (θ F ,θ D The loss function is backpropagated from the discriminator module. Backpropagation of this loss function changes the network parameters (θ) of the preceding feature extractor. F ) and the network parameters (θ) of the conditionally weighted discriminator D The backpropagation of this loss function makes the features extracted by the feature extractor less likely to be judged by the discrimination module to determine whether they come from the source domain or the target domain, thereby achieving the goal of making the features of the source domain and the target domain more similar.

[0126] The above describes the structure of the feature extractor, sample reweighting classifier, and conditional reweighting discriminator in the preset SRENet network, and also provides a brief introduction to the network training process. For an optional structure of the preset SRENet network, please refer to [link to documentation / reference]. Figure 6 In this diagram, the three dashed boxes represent the feature extractor, the sample reweighting classifier, and the conditional reweighting discriminator, respectively. `Conv 1d` represents a one-dimensional multi-channel convolutional layer, where (F1,K) or (2F1,2K) represents (number of kernels, kernel length); `BatchNorm 1d` represents the first or second batch normalization layer; `ReLU` represents the activation function layer; `AvgPool 1d` represents the average pooling layer; `Dropout` represents the random dropout layer; and `Flatten` represents the data flattening layer. `SDM` represents the weight calculation module; `FC` represents the fully connected layer, used as a classification or discriminator module; `classifier loss` represents the classification error; and `Classifier Reweighted Loss` represents the loss function L of the sample reweighting classifier. RC (θ F θ C GRL stands for Gradient Reversal Layer; Domain Loss represents the Conditional Reweighted Discriminator Loss Function L. D (θ F ,θ D ).

[0127] With reference to the embodiments of the present invention, please refer to Figure 6 Understanding how to train a pre-defined SRENet network using multi-source domain sample datasets and target domain sample datasets includes:

[0128] For each training iteration using mini-batch samples, during forward and backward propagation, the predefined total loss function of the SRENet network is minimized, and the parameter θ of the feature extractor is optimized. F The parameters θ of the sample reweighted classifier C Based on the optimized θ F and θ C Maximize the total loss function and optimize the parameters θ of the conditionally reweighted discriminator. D The parameter optimization process iteratively minimizes and maximizes the total loss function; and when the iteration stopping condition is met in the current iteration, the trained target SRENet network is obtained.

[0129] Specifically, for each iteration, a mini-batch of samples is used for training. Training follows the sequence of data input - network forward propagation - loss function calculation - backpropagation to optimize network parameters. During training, the Adam optimization method is used to find the optimal parameters of SRENet, and the parameter θ of the feature extractor is... F The parameters θ of the sample reweighted classifier C The parameters θ of the conditional reweighted classifier D It is being optimized. The first objective is θ. F and θ C By minimizing the loss function L of the sample reweighting classifier C (θ F ,θ C Joint optimization is performed to obtain discriminative features and label predictions. The second objective is θ. D By minimizing the conditionally reweighted discriminant loss function L D (θ F ,θ D To optimize θ, we can improve the discriminator's discrimination ability; and calibrate θ by maximizing it. F This allows for the acquisition of common feature representations between the source and target domains. During training, the sample reweighting classifier reduces the negative impact of overly discrete samples on classification performance; the conditional reweighting discriminator enhances the positive impact of easily transferable samples on domain adaptation. Furthermore, the weight allocation of samples by the sample reweighting classifier and the conditional reweighting discriminator during training constrains the feature extractor during backpropagation, enabling it to extract more general features between the source and target domains, thereby improving robustness and adaptability to data from different domains.

[0130] The expression for the total loss function is:

[0131] L(θ F ,θ C ,θ D ) = L C (θ F ,θ C )-λL D (θ F ,θ D )

[0132] L(θ F ,θ C ,θ D ) represents the total loss function; L C (θ F ,θ C ) represents the loss function of the sample reweighted classifier; L D (θ F ,θ D) represents the loss function of the conditionally reweighted discriminator; λ represents the preset hyperparameter, which is the trade-off between the two objectives during the optimization process.

[0133] In other words, the parameters of the pre-defined SRENet network are obtained by iteratively minimizing and maximizing the total loss function L(θ). F ,θ C ,θ D (to conduct training)

[0134] For the first objective, minimize the loss to update the parameter θ. F and θ C It can be represented as:

[0135]

[0136] In this process, θ D Keeping θ constant, optimize F and θ C ,get and This represents a parameter that is fixed for D.

[0137] Next, regarding the second objective, based on the already optimized... and To maximize the total loss function to update the parameter θ D It can be represented as:

[0138]

[0139] During this process, and Keep it unchanged, only optimize θ D ,get

[0140] In this embodiment of the invention, a gradient inversion layer is inserted between the feature extraction layer and the conditional reweighted discriminator. Figure 6 (GRL in training). During forward propagation, GRL has no effect on the gradient; during backpropagation, GRL reverses the gradient by multiplying by a negative scalar -λ. By using backpropagation, a sample-weighted classifier, and a conditionally weighted discriminator, the classification performance for the target subjects can be further improved. Furthermore, during backpropagation in the training process, the sample-weighted classifier assigns higher weights to source domain samples with higher weights. The conditionally weighted discriminator assigns higher weights to features with higher classification probabilities in domain adaptation.

[0141] S3, use the target SRENet network to obtain the decoding result of the sample to be tested.

[0142] The test samples are obtained by collecting MI EEG signals from the target subjects. The test samples can be target domain samples that participated in network training or target domain samples that did not participate in network training.

[0143] Specifically, the decoding results of the test samples are obtained using the target SRENet network, including:

[0144] The test sample is input into the target SRENet network, and the decoding result of the test sample is obtained by using the feature extractor and the sample reweighting classifier.

[0145] Please see Figure 6 For a given test sample, after inputting it into the target SRENet network, features are extracted using a feature extractor, and then the classification module in the sample reweighted classifier is used. Figure 6 The FC in the sample reweighted classifier classifies the feature and obtains the classification result of the sample to be tested as the decoding result.

[0146] This invention provides a method for cross-device decoding of EEG signals using multi-source domain sample reweighting. It utilizes labeled multi-source domain sample datasets obtained from the subject's MI EEG signals and unlabeled target domain sample datasets to train a pre-constructed SRENet network, resulting in a trained target SRENet network. The target SRENet network is then used to obtain the decoding results for the test samples. The SRENet network in this invention includes a feature extractor, a sample reweighting classifier, and a conditional reweighting discriminator. It establishes a MI EEG transfer learning network between the multi-source domain and a target domain, and is also a sample reweighted EEG decoding network. A sample reweighting strategy is employed to improve cross-device MI EEG decoding performance. During network training, the sample reweighting classifier and conditional reweighting discriminator measure the transferability of features extracted by the feature extractor. Source domain samples that are more easily transferred to the target domain are assigned higher training weights. Therefore, it can effectively utilize data from the multi-source domain data that is beneficial for target domain classification for effective training, further improving the cross-device MI EEG signal decoding performance through a novel transfer learning method. Furthermore, the weight allocation of samples by the sample reweighting classifier and the conditional reweighting discriminator during training plays a constraining role on the feature extractor during backpropagation, enabling the feature extractor to extract more general features for both the source and target domains. This improves the robustness and adaptability to data from different domains.

[0147] Furthermore, in the network training process, this embodiment of the invention automatically reweights all source domain data based on the feature distance and classification confidence between the source and target domains, thereby accurately predicting the labels of the target domain data. It can effectively utilize MI EEG data from multiple source domains to train cross-device decoding algorithms, which has significant application value for deploying decoding algorithms on new devices.

[0148] To verify the effectiveness of the method in the embodiments of the present invention, experimental data are used for illustration below.

[0149] 1. The MI EEG dataset used in the experiment

[0150] To verify the performance of the method proposed in this embodiment of the invention, namely, using the target SRENet network for MI (RRG decoding) (hereinafter referred to as SRENet) on cross-device MI EEG decoding tasks, this embodiment of the invention conducted experiments on three public datasets: the GigaScience dataset, and BCI Competition IV datasets 2a and 2b.

[0151] (1) The GigaScience database (Giga for short)

[0152] The dataset consists of 64 channels of EEG signals from 52 subjects, involving a two-class MI task involving the left and right hands. EEG data were acquired using 64 Ag / AgCl active electrodes. The dataset includes either 100 or 120 trials for each subject. The EEG signals were sampled at a frequency of 512 Hz.

[0153] (2) Dataset 2a of the BCI Competition IV (abbreviated as 2a)

[0154] The dataset consists of 22 channels of EEG signals from 9 subjects, covering four categories of tasks: left hand, right hand, foot, and tongue-related (MI). EEG data were acquired using 22 Ag / AgCl active electrodes. The dataset contains 576 trials for each subject. The EEG signals were sampled at a frequency of 250 Hz.

[0155] (3) Dataset 2b of the BCI Competition IV (abbreviated as 2b)

[0156] The dataset consists of three channels of EEG signals from nine subjects, involving a two-class MI task involving left and right hands. The dataset contains 720 trials for each subject. The EEG signals were sampled at a frequency of 250 Hz.

[0157] In the following experiments, the Giga dataset was selected with a time interval of [0s, 3s] after the start of the cue in each trial, and the 2a and 2b datasets were selected with a time interval of [1s, 4s]. The same 22-channel approach was used in the Giga to 2a trials, and the same 3-channel approach was used in both the Giga to 2a and 2a to 2b tasks. To ensure comparability of results, only left-handed imagination classification was used. Furthermore, the EEG signals were preprocessed using a causal third-order Butterworth filter in the (4–38Hz) frequency band.

[0158] 2. Experimental Setup

[0159] The SRENet described in this embodiment of the invention is compared with seven existing algorithms related to the field: FBCSP, ConvNet, EEGNet, MSFBCNN, DDC, DANN, and CDAN.

[0160] The performance of all the above methods was evaluated using classification accuracy, and the hyperparameters of SRENet were set to F1=16 and K=31.

[0161] 3. Experimental Results

[0162] MI EEG decoding experiments were conducted across different datasets (corresponding to cross-device tasks). For Giga, data from the top 9 subjects were selected; for 2a and 2b, data from all 9 subjects were selected. To simulate real-world applications, three experiments were conducted in this embodiment: Giga to 2a, Giga to 2b, and 2a to 2b, representing cross-device tasks from multi-channel to few-channel. In all three tasks, the common channels between each pair of datasets were used. In each task, a decoding algorithm was trained using one dataset to adapt to all object data in the other dataset. For example, to implement the Giga to 2a task, data from Giga was used to adapt to the data from each subject in 2a. The performance of the method and several comparative algorithms in this embodiment on the three tasks is shown below. Table 1 shows the classification accuracy of each decoding method in the Giga to 2a task. Methods represents each method, S1 to S9 under Target Subjects represent 9 subjects, and Average represents the average value.

[0163] Table 1. Classification accuracy (%) of the Giga to 2a migration task.

[0164]

[0165] Experimental results show that SRENet in this embodiment of the invention has higher accuracy than all the comparison methods. Even compared with EEGNet, the most popular deep learning method in decoding, the classification accuracy of the method in this embodiment of the invention is improved by an average of 6.88%. Furthermore, as shown in Table 1, the domain adaptation methods DDC and DANN do not achieve significant improvements over the classic deep learning methods ConvNet, EEGNet, and MSFBCNN. This means that traditional domain obfuscation methods cannot be well applied to multi-source domain data across devices. The conditional domain adaptation method CDAN has significantly better accuracy than other domain adaptation methods, and the results show that CDAN's conditional discriminator has a positive effect on MIEEG inter-device transmission tasks. It also proves the effectiveness of the SRENet design, where the conditional discriminator is a reweighted method.

[0166] Table 2. Classification accuracy (%) of the Giga to 2b migration task.

[0167]

[0168]

[0169] Table 2 shows the classification accuracy for the Giga to 2b task. In this task, SRENet still achieves the best performance. Compared to EEGNet, SRENet improves classification accuracy by 5.90%. However, other domain adaptation methods do not show a significant advantage over traditional deep learning methods, which is uncommon in cross-device decoding tasks. This may be because data samples across devices are excessively dispersed across multiple domains, causing comparison domain adaptation methods to fail. Furthermore, although SRENet's architecture includes a conditional domain adaptation component, SRENet's performance is still significantly better than CDAN, demonstrating the effectiveness of SRENet's sample reweighting classifier.

[0170] Table 3. Classification accuracy (%) of the migration task from 2a to 2b.

[0171]

[0172] Table 3 shows the classification accuracy for tasks 2a to 2b. In Table 3, SRENet still achieves the best performance among all methods, outperforming EEGNet by an average of 3.49%. However, the domain adaptation methods DAN and CDAN perform worse than all traditional deep learning methods. This may be because both the source and target data have only a few channels, which makes domain adaptation methods ineffective, especially with multi-source domain data. This means that for simple data, traditional domain adaptation methods may fail and cannot effectively obfuscate the source and target domains. In contrast, SRENet's sample reweighting strategy remains effective, demonstrating the robustness of the SRENet architecture.

[0173] 4. Results Analysis

[0174] (1) Statistical Analysis

[0175] The average results of all comparison methods across the three transmission tasks are as follows: Figure 7 As shown, Figure 7 This is a comparison chart of the average accuracy of each comparison method on three transmission tasks in the experiments of this embodiment of the invention.

[0176] Each comparison method is represented by three bar charts from left to right, showing the average results of that method on the three tasks Giga-2a, Giga-2b, and 2a-2b, respectively; the line chart represents the average accuracy across different comparison methods. The datasets used for these three tasks were collected using three different devices. Figure 7 As can be seen, the SRENet proposed in this embodiment achieves the best results on all tasks compared to the other comparative methods, proving that the method of this embodiment has good adaptability to inter-device transmission tasks. Furthermore, it can be seen that all deep learning-based methods outperform the traditional FBCSP, demonstrating the advantage of deep learning-based methods in feature extraction. However, the deep learning-based domain adaptation methods DDC, DAN, and CDAN do not show any advantage over other deep learning methods. In the transfer tasks with only 3 channels in 2a to 2b, the performance of the three domain adaptation methods even decreases. This indicates that existing domain adaptation methods cannot be successfully used for multi-source domain (MI) and EEG domain adaptive decoding tasks. The method of this embodiment has successfully solved this problem; the reweighted training strategy yields satisfactory results regardless of whether more or fewer channels are used.

[0177] Table 4. Statistical significance test of p-values

[0178] Comparisons\Tasks Giga-2a Giga-2b 2a-2b FBCSP 2.30E-04 3.35E-04 1.46E-05 ConvNet 1.88E-04 9.73E-04 8.01E-03 EEGNet 1.34E-03 4.05E-04 2.52E-02 MSFBCNN 9.78E-05 1.67E-03 1.21E-02 DDC 5.96E-04 1.90E-03 3.18E-02 DANN 6.72E-04 7.35E-04 3.25E-03 CDAN 2.33E-03 2.97E-03 1.31E-02

[0179] To further verify the statistical significance of the proposed SRENet, a set of t-tests were performed between SRENet and all comparison methods. The t-test, also known as the Student's t-test, is primarily used for normally distributed samples with small sample sizes (e.g., n < 30) and unknown population standard deviation σ. The t-test uses t-distribution theory to infer the probability of differences, thereby comparing whether the difference between two means is significant. It is mainly used to indicate whether there is a significant difference between the two sets of results, that is, whether the new result is significantly better than the existing result. The t-test results are shown in Table 4. If the p-value of a pairwise t-test is less than 0.05, a significant difference between the two comparison methods can be confirmed. Here, p is the probability of committing a Type I error, i.e., the probability of the null hypothesis being true and rejected; it is generally controlled to be less than 0.05. In other words, a p-value less than 0.05 indicates that the result is significant. The p-values ​​in Table 4 show that the method of this embodiment is significantly better than all other comparison methods. Furthermore, it can be clearly seen that the p-value in the Giga to 2a task is lower than that in the other two tasks. This demonstrates that SRENet has a greater advantage over memory-based deep learning methods and domain adaptation methods in multi-channel transfer tasks. In summary, statistical analysis proves the advancements of SRENet in inter-device MI EEG decoding tasks.

[0180] (2) Feature visualization analysis

[0181] To reveal the impact of network architecture on feature extraction, the T-SNE method was used to visualize the features extracted by SRENet (SRENet-base) without sample reweighting classifiers and conditional reweighting discriminators, and by standard SRENet (SRENet) with sample reweighting classifiers and conditional reweighting discriminators. The Giga-2a transfer learning task was performed, where the Giga dataset represents the multi-source domain and subject 8 represents the target data. In this task, the classification accuracies of SRENet-base and SRENet were 75.50% and 84.38%, respectively. The feature visualization results are labeled with domain labels and classification labels, respectively. The feature visualization results are shown below. Figures 8(a) to 8(d) As shown, each figure has been converted to grayscale. Figure 8(a) shows the feature visualization results of SRENet-based feature extraction plus domain labels; Figure 8(b) shows the feature visualization results of SRENet-based feature extraction plus classification labels; Figure 8(c) shows the feature visualization results of SRENet feature extraction plus domain labels; and Figure 8(d) shows the feature visualization results of SRENet feature extraction plus classification labels. The results are analyzed from the perspectives of domain and classification respectively as follows:

[0182] 1) From the perspective of domain

[0183] Figures 8(a) and (c) show the feature distribution labeled by domain. The features extracted by SRENet-base shown in Figure 8(a) show no clear relationship between the source and target domains. In SRENet-base, all samples are trained with equal weights, which makes the feature space cluttered. However, it is clear that the features extracted by SRENet shown in Figure 8(c) exhibit significant distance relationships between different source and target domains. For example, subject 4 in the source domain is significantly closer to the target domain, while subject 7 in the source domain is furthest from the target domain in the feature space. This distance relationship between features reflects the effectiveness of the sample reweighting classifier during training. By reweighting samples, samples with low distances to the target domain in the feature space are processed together, helping the network ignore overly scattered samples in the source domain and improving classification performance.

[0184] 1) From a classification perspective

[0185] Figures 8(b) and (d) show the feature distribution labeled with classification tags. The SRENet-based features shown in Figure 8(b) lack clear classification boundaries across multiple source domains, meaning that during training, the network cannot learn an effective mapping between source data and features from potentially conflicting source data. Figure 8(d) shows the features extracted by SRENet. In the visualization results, source domain samples closer to the target domain perform better in classification than distant samples. These closer samples mostly come from the closer source domains shown in Figure 8(c), demonstrating that the sample reweighting classifier and conditional reweighting discriminator jointly drive SRENet to select these source domains and samples, facilitating target domain classification during training. Feature visualization experiments demonstrate the contribution of the sample reweighting strategy to the network training process.

[0186] (3) Effectiveness analysis of the sample reweighting strategy

[0187] To investigate the effectiveness of the sample reweighting strategy, an experiment was conducted in this embodiment of the invention to observe the relationship between feature distribution and classification ability of each source domain. First, the inter-domain distance (from each source domain to the target domain) and inter-class distance were measured for each source domain in Figures 8(c) and 8(d). Then, SRENet-base was trained using data from subjects 1-9 (S1-S9) on the Giga dataset, and its performance was tested on data from subject 8 on the 2a dataset. The experimental results are as follows: Figure 9 As shown. Figure 9 This is a graph showing the relationship between the feature distribution and classification ability of each source domain in the experiments of this embodiment of the invention. The bars represent the classification accuracy of SRENet-base; line 1 represents the inter-domain distance from the source domain to the target domain of SRENet; and line 2 represents the inter-class distance from the source domain to the target domain of SRENet.

[0188] In this experiment, it is clearly shown that the classification accuracy of single-source domain training using SRENet-base is significantly correlated with the inter-domain and inter-class distances of multi-source domain training using SRENet. High-precision source domains such as S4, S1, and S3 in the SRENet-base training task have low inter-domain distances and high inter-class distances in the SRENet training task. Conversely, low-precision source domains such as S2, S9, S6, S7, and S8 in the SRENet-base training task have relatively high inter-domain distances and low inter-class distances in the SRENet base training task. This means that the sample reweighting strategy correctly focuses on these transferable source domains and assigns them higher training weights. Furthermore, S5 performs the worst in the SRENet-base training task and has the lowest inter-class distance in the SRENet training task. Although the features of S5 are close to the target domain, it is not a transferable domain from a classification perspective. Therefore, the conditional reweighted discriminator assigns lower weights to S5 during training. As a result, the sample reweighting classifier and the conditional reweighting discriminator worked correctly in the experiment, proving the effectiveness of the sample reweighting strategy.

[0189] In summary, to improve the decoding performance of inter-device motion imaging (MI) electroencephalography (EEG), this invention proposes a sample reweighting EEG decoding network (SRENet) based on a sample reweighting strategy. SRENet employs a sample reweighting classifier and a conditional reweighting discriminator, assigning higher training weights to samples that are easily transmitted between devices. To evaluate the performance of SRENet, this invention conducted three inter-device transmission tasks using the GigaSceene dataset and BCI Competition IV datasets 2a and 2b. The results show that SRENet significantly improves classification accuracy compared to comparative methods, demonstrating the effectiveness of the sample reweighting strategy. This provides a novel solution for inter-device MI EEG decoding applications.

[0190] The above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for cross-device decoding of EEG signals with multi-source domain sample reweighting, characterized in that, include: A multi-source domain sample dataset is obtained by collecting MI EEG signals from multiple subjects, and a target domain sample dataset is obtained by collecting MI EEG signals from a target subject; wherein, the multi-source domain sample dataset includes multiple source domain samples and corresponding classification labels; the target domain sample dataset includes multiple target domain samples; The pre-defined SRENet network is trained using the multi-source domain sample dataset and the target domain sample dataset to obtain the trained target SRENet network. The pre-defined SRENet network includes a feature extractor, a sample reweighting classifier, and a conditional reweighting discriminator. During network training, the transferability of the features extracted by the feature extractor is measured using the sample reweighting classifier and the conditional reweighting discriminator, and source domain samples that are more easily transferred to the target domain are assigned higher training weights. The decoding result of the test sample is obtained using the target SRENet network; wherein the test sample is obtained by collecting the MI EEG signal of the target subject; The sample reweighting classifier includes: The weight calculation module is used to determine, during network training, the distance between the features of each source domain sample and the center of the target domain feature space, as well as the dispersion of the features of the source domain samples and the features of the target domain samples, in the corresponding mini-batch samples based on the features of the source domain samples and the target domain samples output by the feature extractor, and to calculate the weight of the corresponding source domain sample using each distance and the dispersion. The classification module is used to classify the features of each source domain sample and each target domain sample during network training to obtain the corresponding classification results; The source domain sample error calculation module is used to obtain the classification error of each source domain sample based on the classification results and corresponding classification labels of each source domain sample during network training. The sample reweighted classifier loss function calculation module is used to weight the classification error of each source domain sample during network training, and obtain the value of the sample reweighted classifier loss function by averaging the weighted results of all source domain samples.

2. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 1, characterized in that, The feature extractor includes: The layers are connected in sequence: the first convolutional layer, the first batch normalization layer, the second convolutional layer, the second batch normalization layer, the activation function layer, the average pooling layer, the random discarding layer, and the data flattening operation layer. The feature extractor is used to extract features from source domain samples and target domain samples in each mini-batch of input samples; the first convolutional layer and the second convolutional layer are both one-dimensional multi-channel convolutional layers.

3. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 2, characterized in that, The conditional reweighting discriminator includes: The conditional feature calculation module is used during network training to combine the features of each sample in the source domain and the target domain with the classification result of the samples in that domain output by the classification module as a condition of the features of the samples in that domain output by the feature extractor. The conditional features of each sample in that domain are obtained by combining the features and conditions of each domain. Gradient inversion layers are used during network training to leave the forward propagation process unchanged, while inverting the gradient during the back propagation process by multiplying by a negative scalar. The discrimination module is used to determine the domain category of each source domain sample and each target domain sample based on the corresponding conditional features during network training, and obtain the corresponding domain discrimination results, so as to confuse the source domain and the target domain during backpropagation and make the features of the two domains more similar. The conditional reweighted discriminator loss function calculation module is used to calculate the value of the conditional reweighted discriminator loss function during network training, based on the domain discrimination results of each source domain sample and each target domain sample output by the discrimination module, and the conditional features of each sample in each domain.

4. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 3, characterized in that, The step involves determining the distance between the features of each source domain sample and the center of the target domain feature space, as well as the dispersion of the features of the source domain samples and the features of the target domain samples, based on the features of the source domain samples and the target domain samples output by the feature extractor. The weight of the corresponding source domain sample is then calculated using these distances and the dispersion. Based on the features of all target domain samples output by the feature extractor Calculate the center of the target domain feature space in the corresponding mini-batch samples. ; Features for each source domain sample output by the feature extractor Calculate the features of the source domain sample. With the center of the target domain feature space distance ; By using the distances between the features of all source domain samples and the center of the target domain feature space, the dispersion of the features of the source domain samples and the features of the target domain samples in the corresponding mini-batch is calculated. ; For each source domain sample, use the distance of that source domain sample. and the degree of dispersion The weights of the source domain samples are calculated using a preset weight calculation formula. The preset weight calculation formula is as follows: The multi-source domain sample dataset is defined as follows: The target domain sample dataset is defined as follows: ; Represents the source domain; Represents the target domain; Indicates the first One source domain sample; express Category tags; This indicates the number of samples in the multi-source domain sample dataset; Indicates the first One target domain sample; This indicates the number of samples in the target domain sample dataset; Indicates the first The weights of each source domain sample.

5. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 4, characterized in that, The calculation formula used to obtain the classification error of each source domain sample based on the classification results and corresponding classification labels of each source domain sample includes: in, Indicates the first The classification error of each source domain sample; Indicates the parameters of the feature extractor; This represents the parameters of the sample reweighting classifier; Indicates the first Classification labels for each source domain sample; Indicates the first Classification results of the source domain samples; Indicates the first category in a classification result Data categories; This indicates the total number of data categories in the classification results.

6. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 5, characterized in that, The step of combining the features and conditions of the domain to obtain the conditional features of each sample in the domain includes: For each feature, perform a multilinear mapping operation between the condition of that feature and the feature itself to obtain a conditional feature matrix for that feature: in, The conditional feature matrix representing a sample; This indicates the classification result of the sample. Each element in the table represents the confidence level for the corresponding data category. This indicates the total number of data categories in the classification results; This represents the characteristics of the sample; Dimensions representing features; This indicates a multilinear mapping operation.

7. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 6, characterized in that, The expression for the conditional reweighted discriminator loss function includes: in, Indicates the first Classification results of the source domain samples; Indicates the first Conditional feature matrix of each source domain sample; Indicates the first Classification results of samples from the target domain; Indicates the first The conditional feature matrix of each target domain sample; This represents the domain discrimination result of the discrimination module on the conditional feature matrix of a sample; Indicates sample weights; This represents the entropy criterion.

8. The method for cross-device decoding of EEG signals with multi-source domain sample reweighting according to any one of claims 1 to 7, characterized in that, The step of training a pre-defined SRENet network using the multi-source domain sample dataset and the target domain sample dataset includes: For each training iteration using mini-batch samples, during forward and backward propagation, the total loss function of the preset SRENet network is minimized to optimize the parameters of the feature extractor. and the parameters of the sample reweighted classifier Based on the optimized and Maximize the total loss function and optimize the parameters of the conditional reweighted discriminator. The parameter optimization process iteratively minimizes and maximizes the total loss function; and when the iteration stopping condition is met in the current iteration, the trained target SRENet network is obtained; The expression for the total loss function is as follows: Represents the total loss function; This represents the loss function of the sample reweighting classifier; This represents the loss function of the conditionally reweighted discriminator; This represents the preset hyperparameters.

9. The cross-device decoding method for EEG signals with multi-source domain sample reweighting according to claim 1, characterized in that, The decoding result of the sample to be tested obtained using the target SRENet network includes: The test sample is input into the target SRENet network, and the decoding result of the test sample is obtained by using the feature extractor and the sample reweighting classifier.

Images