A motor imagery recognition method and system based on cross-subject small sample learning

Abstract

The present invention discloses a method for identifying motor imagery with cross-subject small sample learning, which is characterized by comprising the following steps: S1: obtaining EEG signals of multiple subjects, dividing them into training sets and target domain data, and performing data preprocessing; S2: collectively referring to the training set and the target domain data as a support set, and using data enhancement technology to generate a mixed set for the training set and the support set; S3: training an initial model using a large amount of labeled data in a pre-training stage; S4: fine-tuning the pre-training model using the mixed set and the support set through a dual loss function; S5: converging the total loss function value through iterative training to obtain a motor imagery cross-subject recognition model with high accuracy for the target subject. A motor imagery recognition system with cross-subject small sample learning is also disclosed. The present invention can significantly improve the classification accuracy of motor imagery across subjects by generating representative synthetic data and combining it with target domain data for training.

Description

Technical Field

[0001] The present invention relates to the technical field of electroencephalogram signal recognition, and in particular to a motor imagery recognition method and system for cross-subject small sample learning. Background Art

[0002] Brain-computer interface (BCI) technology establishes a direct connection between the human brain and external devices. This technology has shown profound value in medical assistance, entertainment, and education. In the field of brain-computer interface (BCI), electroencephalography (EEG) is widely used due to its non-invasiveness and high temporal resolution. Motor imagery (MI) associated with sensorimotor rhythms (SMR) is a prominent and widely studied EEG-based BCI paradigm. This paradigm allows users to mentally imagine specific actions without performing actual actions without the involvement of physical muscles. Precisely, decoding motor imagery EEG (MI-EEG) signals can extract the patterns of MI, which can then be converted into control commands for various external devices, such as exoskeletons, wheelchairs, and computers.

[0003] In the field of traditional MI-BCI systems, manual features are usually extracted from MI-EEG signals and classifiers are constructed using machine learning methods. Many studies have explored different feature extraction methods to improve recognition accuracy and achieved good results. However, the above methods rely heavily on the utilization of manual features. Manual feature extraction not only requires a lot of domain expertise, but is also very time-consuming. In addition, these methods are very susceptible to various noise sources, including physiological, electronic equipment, and environmental noise.

[0004] Due to the remarkable effectiveness of CNN, several well-known MI-EEG decoding models have been built around this architecture. For example, EEGNet stands out for its efficient and robust performance; ConvNet is known for its ability to effectively capture time-related dynamics and spatial features in the data. In addition, the innovative Conformer combines the advantages of CNN and Transformer, standing out through this novel fusion. Although these deep learning methods have achieved excellent results, there is a major issue that has not been considered. In fact, due to the variability of the oscillation patterns of brain activity, classification models built based on existing subjects will encounter difficulties when applied to new target subjects. Although building a separate classifier for each user is a solution, it is inefficient and time-consuming, and it is difficult to implement in practical applications.

[0005] Transfer learning provides an effective solution to enhance the robustness of different subjects. Transfer learning methods aim to transfer knowledge learned from other source subject domains to the target subject domain, thereby improving the recognition accuracy of the target subject. There are many transfer learning methods that have been widely used in MI-EEG decoding. However, transfer learning methods require a large amount of sample data from the target subject for model calibration. This limitation leads to inevitable mental fatigue of the subject and poor initial recognition accuracy, resulting in a poor user experience.

[0006] Therefore, there is an urgent need to provide a new motor imagery recognition method and system based on small sample learning across subjects to solve the above problems. Summary of the invention

[0007] The technical problem to be solved by the present invention is to provide a method and system for motor imagery recognition with small sample learning across subjects, which combines synthetic data generation and a dual loss function, generates representative synthetic data through data enhancement, and combines it with target domain data for training, which significantly reduces the overfitting problem of the running imagination model and can significantly improve the classification accuracy of motor imagery across subjects.

[0008] In order to solve the above technical problems, a technical solution adopted by the present invention is to provide a method for motor imagery recognition by learning across a small sample of subjects, comprising the following steps:

[0009] S1: Divide the EEG signals of multiple subjects into source domain data and target domain data and perform data preprocessing;

[0010] S2: The source domain data is called the training set, and a small amount of target domain data is called the support set. A subset of the training set and the support set are randomly selected for mixing, classification, segmentation, and reconstruction to obtain a mixed set.

[0011] S3: Use the training set to train the initial model in the pre-training stage to obtain a highly robust pre-training model;

[0012] S4: fine-tuning the parameters of the pre-trained model using a dual loss function with a regularization technique method using the mixed set and the support set, so that the pre-trained model can have a better classification accuracy for the required target subject;

[0013] S5: Through iterative training, the total loss function value of the dual loss function converges, and a flooding strategy is introduced to prevent the training loss from being excessively reduced, thus obtaining a motor imagery cross-subject recognition model with high accuracy for the target subjects.

[0014] In a preferred embodiment of the present invention, in step S1, the data preprocessing process includes:

[0015] Use a sixth-order Chebyshev bandpass filter to perform bandpass filtering on the EEG signal in the range of [4,40] Hz to obtain the EEG signal in the frequency band related to motor imagery and remove power frequency interference and other high and low frequency noises;

[0016] The preprocessed EEG signal is segmented to obtain time segments of fixed length to obtain the training set and the target domain data set.

[0017] In a preferred embodiment of the present invention, the specific steps of step S2 include:

[0018] S201: First, the data set except the target subject is used as the training set, then a small amount of target subject data is used as the support set, and finally a subset of the training set is randomly selected, and the subset is merged with the support set to obtain a mixed set;

[0019] S202: In the data set synthesized in step S201, eight trials are selected from each category, and new trials are constructed using the eight trials and the segmentation and reconstruction processes at different time points, designated as , the first eighth of the first trial is used as The first part of the second trial constitutes the second eighth of The subsequent part of the branch, and so on, all eight branches continue this pattern in sequence, each The combinations are all combinations of fragments from eight different experiments. The associated tag is named , where y represents the label of the imagined movement part during motor imagery, which is consistent with the labels of the eight trials constructed by it;

[0020] S203: In order to maintain balance, this segmentation and reconstruction process is repeated multiple times for each category of motor imagery EEG data MI-EEG. After repetitions, the number of trials in the mixed set is the same as that in the support set.

[0021] In a preferred embodiment of the present invention, the specific steps of step S3 include:

[0022] S301: inputting all MI-EEG test data in the training set into the pre-training model, and outputting the probability distribution of the predicted label through the embedding layer and the classification layer in the pre-training model;

[0023] S302: Apply the cross entropy loss function to calculate the difference between the predicted label and the actual label, and train the initial parameters of the pre-trained model by minimizing the loss, so as to obtain a general model with higher robustness.

[0024] In a preferred embodiment of the present invention, the specific steps of step S4 include:

[0025] Fine-tune the pre-trained model using the mixed set and the support set, and optimize the parameters of the pre-trained model by minimizing the dual loss function;

[0026] During the fine-tuning process, a combination of the maximum mean difference loss function and the cross entropy loss function with a regularization technique is used. The maximum mean difference loss function is used to reduce the distribution difference between the training set and the target domain data, and by minimizing the maximum mean difference loss MMD, the model can better adapt to the differences between different subjects; the cross entropy loss function is used to calculate the difference between the predicted label and the actual label, and by minimizing the cross entropy loss CE, the classification accuracy of the model is ensured.

[0027] In a preferred embodiment of the present invention, in step S5, through multiple iterations of training, the model parameters are gradually adjusted so that the total loss function value is continuously reduced until convergence; by introducing a flooding strategy, the training loss is prevented from being excessively reduced, and the loss function is adjusted by setting the flooding level to improve the stability and generalization ability of the model; in each iteration, the total loss value of the current model on the test set is calculated, including the maximum mean difference loss MMD and the cross entropy loss CE, and the model parameters are adjusted according to the change of the total loss value; when the total loss value converges to a stable state, the training ends, and the final motor imagery cross-subject recognition model is obtained.

[0028] In order to solve the above technical problems, another technical solution adopted by the present invention is to provide a motor imagery recognition system for learning across a small sample of subjects, comprising:

[0029] A data acquisition and preprocessing module is used to divide the EEG signals of multiple subjects into source domain data and target domain data and perform data preprocessing;

[0030] The synthetic data generation module is used to call the source domain data a training set and a small amount of target domain data a support set, randomly select subsets of the training set and the support set for mixing, classification, segmentation, and reconstruction to obtain a mixed set;

[0031] The pre-training model module is used to train the initial model using the training set in the pre-training phase to obtain a highly robust pre-training model;

[0032] A model fine-tuning module, used to fine-tune the parameters of the pre-trained model using a mixed set and a support set through a dual loss function with a regularization technique method, so that the pre-trained model can have a better classification accuracy for the required target subjects;

[0033] The iterative training and output module is used to converge the total loss function value of the dual loss function through iterative training, introduce a flooding strategy to prevent the training loss from being excessively reduced, and obtain a motor imagery cross-subject recognition model with high accuracy for the target subjects.

[0034] In a preferred embodiment of the present invention, the model fine-tuning module includes:

[0035] The domain alignment unit is used to introduce the maximum mean difference loss function and the cross entropy loss function to align the inter-domain distribution by minimizing the difference in marginal distribution after the mixture set and the support set are embedded, and the conditional distribution between the mixture set and the support set is aligned through the label prediction output.

[0036] In a preferred embodiment of the present invention, the iterative training and output includes:

[0037] The flooding regularization unit is used to introduce a flooding strategy to prevent excessive reduction of training loss. The loss function is adjusted by setting the flooding level to improve the stability and generalization ability of the model.

[0038] The beneficial effects of the present invention are:

[0039] (1) The present invention further demonstrates its excellent performance and broad application prospects. First, the data enhancement technology of the present invention can effectively expand the small sample data set, so that the model can be exposed to more diverse data during the training process, thereby improving the generalization ability and robustness of the model;

[0040] (2) Secondly, the dual loss function design proposed in the present invention not only takes into account the classification accuracy, but also emphasizes the robustness of feature representation. By optimizing these two objectives at the same time, the present invention can improve the adaptability of the model to the motor imagery patterns of different subjects while ensuring recognition accuracy. This dual loss function design concept also provides a valuable reference for other cross-subject or cross-domain learning tasks;

[0041] (3) In practical applications, the method and system of the present invention have significant advantages. First, it can significantly reduce the reliance on a large amount of labeled data of the target subjects, which is particularly important for those fields where it is difficult to obtain a large amount of labeled data. Second, after training, the model of the present invention can quickly and accurately identify the motor imagery patterns of different subjects, providing effective technical support for the fields of motor rehabilitation, human-computer interaction, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 It is a flow chart of the motor imagery recognition method for learning across a small sample of subjects of the present invention;

[0043] Figure 2 is a diagram of the formation process of the mixed set;

[0044] Figure 3 is a schematic diagram of the overall model of the motor imagery recognition method for learning across small samples of subjects;

[0045] Figure 4 It is a schematic diagram of the effect of different values ​​of the mixing amount on the model recognition accuracy on the IV-2a and IV-2b data sets during the generation of the mixed set;

[0046] Figure 5 It is a schematic diagram of the effect of different values ​​of the flooding scale on the model recognition rate accuracy on the IV-2a and IV-2b datasets during the iterative training process;

[0047] Figure 6 is a schematic diagram comparing the accuracy of different network models on different data sets with and without the method of the present invention;

[0048] Figure 7 It is a structural block diagram of the motor imagery recognition system for cross-subject small sample learning. DETAILED DESCRIPTION

[0049] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

[0050] See also Figure 1 , the embodiment of the present invention includes:

[0051] A method for motor imagery recognition based on cross-subject small sample learning includes the following steps:

[0052] S1: Divide the EEG signals of multiple subjects into source domain data and target domain data and perform data preprocessing;

[0053] It should be noted that EEG signals are easily affected by people’s psychological state and equipment noise, which makes the data distribution of the training set and the test set inconsistent. is the training set, the target domain is the test set. In this embodiment, the classification task of motor imagery EEG signals is four categories, including imagining the movement of the left hand, right hand, foot and tongue, each of which has a specific label. For a subject, the data of the current subject is used as the target domain, and the data from other subjects is used as the source domain.

[0054] Wherein, the pretreatment process includes:

[0055] S101: Use a sixth-order Chebyshev bandpass filter to perform bandpass filtering on the EEG signal in the range of [4,40] Hz to obtain the EEG signal in the frequency band related to motor imagery and remove power frequency interference and other high and low frequency noise;

[0056] S102: Segment the preprocessed EEG signal to obtain time segments of fixed length to obtain a training set and a target domain data set.

[0057] S2: The source domain data is called the training set, and a small amount of target domain data is called the support set. Subsets of the training set and the support set are randomly selected for mixing, classification, segmentation, and reconstruction to obtain a mixed set. Specifically, segmentation and reconstruction (S&R) is used in the time domain to synthesize new data.

[0058] First, it is assumed that there are enough labeled datasets from other subjects (denoted as training sets). However, for the target subject, only a small number of labeled datasets are available for each class, and these trials constitute an initial sample set, called the support set. The model trained on trials that have not been seen by the target subject (called the test set) and the support set are evaluated here. In the present invention, all three datasets have the same label space, denoting the number of classes as C, and the size of C is 4. In the support set, each class has the same number of datasets, denoted as K. This strategy is called C-way, K-shot. The set of MI-EEG trials in the support set is denoted as ,here represents the MI-EEG data of the ith trial in the support set, where E represents the number of electrodes, T represents the time point, Support set The labels of trials are m, where m is the total number of classes. The total number of trials in the support set is given by Similarly, other sets can also be represented by and Specific formulas.

[0059] S201: Use the data set other than the target subject as the training set, and then use a small amount of the target subject's data as the support set. In order to generate a mixed set, the training set and the support set need to be combined first. However, considering the huge difference in the number of trials between the training set and the support set, a subset of the training set is randomly selected, and the number of trials of the subset is defined as a hyperparameter called the mixing amount. Then merge the subset with the support set;

[0060] S202: Figure 2 As shown in the synthetic dataset, eight trials are selected from each category, where the trial refers to the EEG data generated during a motor imagery process. These eight trials are used to construct new trials using the segmentation and reconstruction process at different time points, designated as The first eighth of the first trial is used as The first part of the second trial constitutes the second eighth of The subsequent part of , and so on, all eight branches continue this pattern in sequence. The combinations were all combinations of segments from eight different trials. The associated tag is named , where y represents the label of the imagined movement part during motor imagery, which is consistent with the labels of the eight trials constructed by it.

[0061] S203: To maintain balance, this segmentation and reconstruction process was repeated multiple times for each category of motor imagery EEG data (MI-EEG). After repetitions, the number of trials in the mixed set was the same as that in the support set.

[0062] S3: Use the training set to train the initial model in the pre-training stage to obtain a highly robust pre-training model;

[0063] like Figure 3 As shown in the figure, the mapping layer includes several operations such as convolution, batch normalization, activation, pooling, dropout, etc. The classification layer includes normalization operations and linear layers.

[0064] The network (initial model) starts with a 2D convolutional layer with 1 input channel, 40 output channels, kernel size (1,25) and stride (1,1), which performs a temporal convolution operation on the input MI-EEG data. Subsequently, it is followed by a 2D convolutional layer with 40 channels both in input and output, which utilizes a kernel dimension of (channels,1) and stride (1,1), where channels corresponds to the number of electrode channels in the motor imagery data. This layer acts as a spatial filter to record the interaction between several electrode channels. Batch normalization is then applied to normalize the output of the convolutional layer. To introduce non-linearity, the exponential linear unit (elu) activation function is used. The model applies an average pooling layer along the temporal axis, which utilizes a convolutional kernel of (1,75) and strides with a stride of (1,15). After the average pooling layer, a dropout layer is introduced, followed by an additional 2D convolutional layer. This layer has 40 input and output channels each, and the kernel size and stride are set to (1,1).

[0065] S301: Initially, all MI-EEG trials in the training set were used for pre-training (e.g. Figure 3 brown line). For each trial in the training set, ,in represents each trial in the training set, that is, the EEG data generated by a motor imagery process, Represents the training set, after the embedding part Classification section After getting the output In order to determine the probability distribution of the predicted labels , using the formula:

[0066]

[0067] right Apply the Softmax function.

[0068] S302: Then, The loss function is used to quantify the difference between the predicted labels and the actual labels. The loss function computes the average negative logarithm of the predicted probability across the entire training set, weighted by the actual label distribution:

[0069]

[0070] in, represents the predicted probability of the cth class for the ith trial, represents the number of training samples, represents the actual label of the cth category in the ith trial, m represents the number of categories, and the pre-trained model parameters are denoted as w1.

[0071] S4: fine-tuning the parameters of the pre-trained model by using the mixed set and the support set through a dual loss function, so that the pre-trained model can have a higher classification accuracy for the required target subjects on the basis of having a certain generalization ability;

[0072] The fine-tuning refers to using a combination of a mixed set and a support set as fine-tuning data. This example uses a two-component loss function to fine-tune the model obtained in the first step. First, the training set and the support set need to be mixed, classified, segmented, and reconstructed to obtain a mixed set. The mixed set and the support set are used to fine-tune the model, which is initialized using the pre-trained parameter w1. In order to optimize the fine-tuning process, a two-component loss function with a regularization technique is used.

[0073] S401: First, the two-component loss function is the sum of the maximum mean difference (MMD) loss function and the cross entropy (CE) loss function. The MMD loss function is applied to the output of the mapping part. The mixed set data and the support set data after mapping can be used respectively and This example uses MMD to quantify two marginal distributions and The difference between , minimize this value and use it as the loss function (LMMD). It can be mathematically expressed by the following equation:

[0074] ,

[0075] Where K is the kernel function, which is composed of the feature map Generate, define the inner product in Hilbert space H as .

[0076] S402: The predicted MMD is used as a measure of the difference between different distributions. This is done by evaluating the distance between the sample means of two independent domains (denoted by H) in the Reproducing Kernel Hilbert Space (RKHS). Use s for supp and m for mix to obtain a more concise expression, the equation is:

[0077] ,

[0078] in , represents the function that has passed through the mapping layer network.

[0079] S403: Cross entropy (CE) loss is used to evaluate the output of the model’s label prediction. The formula for cross entropy loss is given above. The only difference is that the model input is specified as , the corresponding true label is In addition, this example Multiply by a factor of 0.1 to ensure that it is consistent with On the same scale. The formula of the composite loss function is as follows:

[0080]

[0081] S5: Through iterative training, the total loss function value of the dual loss function converges, and a flooding strategy is introduced to prevent the training loss from being excessively reduced, thus obtaining a motor imagery cross-subject recognition model with high accuracy for the target subjects.

[0082] Use the parameters w1 obtained from pre-training as the initial parameters of the new model. It is worth noting that the new model has the same network structure as the previously trained model.

[0083] This example uses a dual loss function that combines a mixed set and a support set and a flooding technique to optimize model parameters. The dual loss function is described in steps S402 and S403, and the flooding technique specifically uses a regularization technique called flooding. Flooding deliberately prevents the training loss from decreasing further when it reaches a reasonable small value. This value is called the flooding level. By introducing a flooding strategy, the training loss is prevented from decreasing excessively, and the loss function is adjusted by setting the flooding level to improve the stability and generalization ability of the model. The formula for flooding is:

[0084] ,

[0085] Where b is the flooding level, i.e., the predetermined value. In order to solve the problem that flooding is overly sensitive to parameters, this example proposes a new form of flooding. The flooding level is set to a multiple of the loss value that first appears in training. This multiple is called the flooding scale, which is a hyperparameter that can better solve the problem of model overfitting.

[0086] After optimization according to the formulas described in steps S4 and S5, the model parameters are updated to w2.

[0087] Using the method described in the present invention, experiments were conducted on two public datasets BCI IV IIa (IV-2a) and BCI IVIIb (IV-2b) and a self-collected dataset. In order to quantitatively evaluate the motor imagery recognition results, the average precision was used as an evaluation indicator.

[0088] Table 1

[0089]

[0090] Several experiments were conducted on three different datasets using the method described in the present invention, and the results were compared with some baseline and cutting-edge methods. The baseline methods used included ShallowNet and EEGNet. These two methods are commonly implemented in the field of EEG data analysis using a CNN-based end-to-end framework. In addition, an efficient model called Conformer was used as a baseline model. In order to make these models have a similar experimental paradigm, the present invention fine-tuned them during the experiment and experimented on the open source code to obtain the results of the above method.

[0091] Table 1 shows the classification results of each subject on IV-2a, IV-2b and self-collected datasets, as well as the overall average results. The last column of the table shows the comprehensive average accuracy. For the IV-2a dataset, the model of the method of the present invention performed excellently, with an average accuracy of 68.8%, which is the highest result, surpassing the second best model TERL, whose average accuracy is 65.0%. It is worth noting that the accuracy of the model of the method of the present invention on subject A01 reached 84.8%, and on subject A07 it reached an astonishing 87.5%. On the IV-2b dataset, the method of the present invention once again maintained its leading position, with an average accuracy of 81.9%. In the case of the self-collected dataset, the method still maintained the highest average accuracy of 73.5%. The performance of the present invention was particularly significant on subject A05, reaching 97.3%, and the accuracy on subject A08 was 78%. In all datasets, the method of the present invention showed consistent superiority, proving that it is always superior to other methods in terms of effectiveness.

[0092] like Figure 4 As shown in Figure 1, the example of the present invention explores the number of instances in the training set that participate in the synthesis. Experiments were conducted on two datasets, IV-2a and IV-2b, and it was found that setting the mixing amount to 80 gave the best performance for IV-2a. Similarly, for IV-2b, 60 was the optimal mixing amount. Despite the fluctuations, the results of the two datasets differed by less than 5%. This shows that changes in the mixing amount slightly changed the results of the model. Figure 5 As shown in the figure, it is clear that the model produces the best results with the flood scale adjusted to 0.8 for both datasets. Moreover, the flood scale value has little effect on the accuracy. Specifically, the accuracy changes by less than 5% for IV-2a and less than 3% for IV-2b.

[0093] In order to prove the effectiveness of the examples of the present invention, multiple experiments were conducted. Three deep learning models (EEGNet, ShallowNet and Conformer) were used on the three datasets IV-2a, IV-2b and SCD. Figure 6 As shown, it is obvious that the methods described in the examples of the present invention (EEGNetwith MixDual-Tuning, ShallowNet with MixDual-Tuning, and Conformer with MixDual-Tuning) have consistently enhanced the effectiveness of the models in a series of experiments. In terms of specific accuracy, ShallowNet has the most significant improvements on IV-2a and SCD, with increases of 23.5% and 24.6%, respectively. EEGNet showed the greatest enhancement on IV-2b, with an increase of 18.8%. On average, ShallowNet showed the greatest overall performance improvement in all three datasets, with an average improvement of 21%. In addition, the accuracy of these models was improved by an average of 17.4% in all three datasets and models. The experimental results show that the examples of the present invention have good performance in various neural network structures.

[0094] See also Figure 7 The present invention also provides a motor imagery recognition system for learning across a small sample of subjects, which is characterized by comprising:

[0095] A data acquisition and preprocessing module is used to divide the EEG signals of multiple subjects into source domain data and target domain data and perform data preprocessing;

[0096] The synthetic data generation module is used to call the source domain data a training set and a small amount of target domain data a support set, randomly select subsets of the training set and the support set for mixing, classification, segmentation, and reconstruction to obtain a mixed set;

[0097] The pre-training model module is used to train the initial model using the training set in the pre-training phase to obtain a highly robust pre-training model;

[0098] A model fine-tuning module, used to fine-tune the parameters of the pre-trained model using a mixed set and a support set through a dual loss function with a regularization technique method, so that the pre-trained model can have a better classification accuracy for the required target subjects;

[0099] The iterative training and output module is used to converge the total loss function value of the dual loss function through iterative training, introduce a flooding strategy to prevent the training loss from being excessively reduced, and obtain a motor imagery cross-subject recognition model with high accuracy for the target subjects.

[0100] Wherein, the model fine-tuning module includes:

[0101] The domain alignment unit is used to introduce the maximum mean difference loss function and the cross entropy loss function to align the inter-domain distribution by minimizing the difference in marginal distribution after the mixture set and the support set are embedded, and the conditional distribution between the mixture set and the support set is aligned through the label prediction output.

[0102] The iterative training and output module includes:

[0103] The flooding regularization unit is used to introduce a flooding strategy to prevent excessive reduction of training loss. The loss function is adjusted by setting the flooding level to improve the stability and generalization ability of the model.

[0104] A motor imagery recognition system for cross-subject small sample learning in this example can execute a motor imagery recognition method for cross-subject small sample learning provided by the present invention, can execute any combination of implementation steps of the method example, and has the corresponding functions and beneficial effects of the method.

[0105] It should be noted that: This embodiment proposes a solution based on data enhancement and dual loss function for the problem of motor imagery recognition in small sample learning across subjects. Here, the division of the above-mentioned functional modules is only used as an example for explanation. In practical applications, various functions can be flexibly allocated to different functional modules according to actual needs to achieve diversified division of the internal structure of the system. These functional modules can be independently undertaken by a single execution unit, or two or more execution units can be integrated into one functional module to fully realize their preset functions.

[0106] For those skilled in the art, each module of the above system can be implemented in whole or in part in the form of software, hardware, or a combination thereof. Specifically, these modules can be embedded in or independent of the processor of the computer device in the form of hardware, or can be directly stored in the memory of the computer device in the form of software, so that the processor can call and execute the functional operations corresponding to each module at any time.

[0107] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A motor imagery recognition method with small sample learning across subjects, It is characterized in that The following steps are involved: S1: Divide the EEG signals of multiple subjects into source domain data and target domain data and perform data preprocessing; S2: The source domain data is called the training set, and a small amount of target domain data is called the support set. A subset of the training set and the support set are randomly selected for mixing, classification, segmentation, and reconstruction to obtain a mixed set. The specific steps include: S201: First, the data set except the target subject is used as the training set, then a small amount of target subject data is used as the support set, and finally a subset of the training set is randomly selected, and the subset is merged with the support set to obtain a mixed set; S202: In the data set synthesized in step S201, eight trials are selected from each category, and new trials are constructed using the eight trials and the segmentation and reconstruction processes at different time points, designated as , the first eighth of the first trial is used as The first part of the second trial constitutes the second eighth of The subsequent part of the branch, and so on, all eight branches continue this pattern in sequence, each The combinations are all combinations of fragments from eight different experiments. The associated tag is named , where y represents the label of the imagined movement part during motor imagery, which is consistent with the labels of the eight trials constructed by it; S203: In order to maintain balance, this segmentation and reconstruction process is repeated multiple times for each category of motor imagery EEG data MI-EEG. After repetitions, the number of trials in the mixed set is the same as that in the support set. S3: Use the training set to train the initial model in the pre-training stage to obtain a highly robust pre-training model; S4: fine-tuning the parameters of the pre-trained model using a dual loss function with a regularization technique method using the mixed set and the support set, so that the pre-trained model can have a better classification accuracy for the required target subject; S5: Through iterative training, the total loss function value of the dual loss function converges, and a flooding strategy is introduced to prevent the training loss from being excessively reduced, thus obtaining a motor imagery cross-subject recognition model with high accuracy for the target subjects.

2. The method for motor imagery recognition based on cross-subject small sample learning according to claim 1, It is characterized in that In step S1, the data preprocessing process includes: Use a sixth-order Chebyshev bandpass filter to perform bandpass filtering on the EEG signal in the range of [4,40] Hz to obtain the EEG signal in the frequency band related to motor imagery and remove power frequency interference and other high and low frequency noises; The preprocessed EEG signal is segmented to obtain time segments of fixed length to obtain the training set and the target domain data set.

3. The motor imagery recognition method for cross-subject small sample learning according to claim 1, It is characterized in that The specific steps of step S3 include: S301: inputting all MI-EEG test data in the training set into the pre-training model, and outputting the probability distribution of the predicted label through the embedding layer and the classification layer in the pre-training model; S302: Apply the cross entropy loss function to calculate the difference between the predicted label and the actual label, and train the initial parameters of the pre-trained model by minimizing the loss, so as to obtain a general model with higher robustness.

4. The method for motor imagery recognition based on cross-subject small sample learning according to claim 1, It is characterized in that The specific steps of step S4 include: Fine-tune the pre-trained model using the mixed set and the support set, and optimize the parameters of the pre-trained model by minimizing the dual loss function; During the fine-tuning process, a combination of the maximum mean difference loss function and the cross entropy loss function with a regularization technique is used. The maximum mean difference loss function is used to reduce the distribution difference between the training set and the target domain data, and by minimizing the maximum mean difference loss MMD, the model can better adapt to the differences between different subjects; the cross entropy loss function is used to calculate the difference between the predicted label and the actual label, and by minimizing the cross entropy loss CE, the classification accuracy of the model is ensured.

5. The method for motor imagery recognition based on cross-subject small sample learning according to claim 1, It is characterized in that In step S5, through multiple iterations of training, the model parameters are gradually adjusted to reduce the total loss function value until convergence; by introducing a flooding strategy, the training loss is prevented from being excessively reduced, and the loss function is adjusted by setting the flooding level to improve the stability and generalization ability of the model; in each iteration, the total loss value of the current model on the test set is calculated, including the maximum mean difference loss MMD and the cross entropy loss CE, and the model parameters are adjusted according to the change of the total loss value; when the total loss value converges to a stable state, the training ends and the final motor imagery cross-subject recognition model is obtained.

6. A motor imagery recognition system with cross-subject small sample learning, It is characterized in that include: A data acquisition and preprocessing module is used to divide the EEG signals of multiple subjects into source domain data and target domain data and perform data preprocessing; The synthetic data generation module is used to call the source domain data a training set and a small amount of target domain data a support set, randomly select a subset of the training set and the support set for mixing, classification, segmentation, and reconstruction to obtain a mixed set; the specific steps include: S201: First, the data set except the target subject is used as the training set, then a small amount of target subject data is used as the support set, and finally a subset of the training set is randomly selected, and the subset is merged with the support set to obtain a mixed set; S202: In the data set synthesized in step S201, eight trials are selected from each category, and new trials are constructed using the eight trials and the segmentation and reconstruction processes at different time points, designated as , the first eighth of the first trial is used as The first part of the second trial constitutes the second eighth of The subsequent part of the branch, and so on, all eight branches continue this pattern in sequence, each The combinations are all combinations of fragments from eight different experiments. The associated tag is named , where y represents the label of the imagined movement part during motor imagery, which is consistent with the labels of the eight trials constructed by it; S203: In order to maintain balance, this segmentation and reconstruction process is repeated multiple times for each category of motor imagery EEG data MI-EEG. After repetitions, the number of trials in the mixed set is the same as that in the support set. The pre-training model module is used to train the initial model using the training set in the pre-training phase to obtain a highly robust pre-training model; A model fine-tuning module, used to fine-tune the parameters of the pre-trained model using a mixed set and a support set through a dual loss function with a regularization technique method, so that the pre-trained model can have a better classification accuracy for the required target subjects; The iterative training and output module is used to converge the total loss function value of the dual loss function through iterative training, introduce a flooding strategy to prevent the training loss from being excessively reduced, and obtain a motor imagery cross-subject recognition model with high accuracy for the target subjects.

7. The motor imagery recognition system for cross-subject small sample learning according to claim 6, It is characterized in that The model fine-tuning module includes: The domain alignment unit is used to introduce the maximum mean difference loss function and the cross entropy loss function to align the inter-domain distribution by minimizing the difference in marginal distribution after the mixture set and the support set are embedded, and the conditional distribution between the mixture set and the support set is aligned through the label prediction output.

8. The motor imagery recognition system for cross-subject small sample learning according to claim 6, It is characterized in that The iterative training and output module includes: The flooding regularization unit is used to introduce a flooding strategy to prevent excessive reduction of training loss. The loss function is adjusted by setting the flooding level to improve the stability and generalization ability of the model.

Images