Electroencephalogram signal classification model training method, electroencephalogram signal classification method and device

Abstract

This application provides a method for training an electroencephalogram (EEG) signal classification model, an EEG signal classification method, and an apparatus. The training method includes: acquiring target domain sample data and source domain sample data; the target domain sample data includes the EEG signal of a target object, and the source domain sample data includes the EEG signal of an auxiliary object other than the target object; acquiring an initial classification model to be trained; performing preliminary training on the initial classification model based on the source domain sample data to obtain a trained candidate classification model; and fine-tuning the candidate classification model based on the target domain sample data and the source domain sample data to obtain a trained target classification model; the target classification model is used to classify the EEG signal of the target object. This method solves the problem of poor versatility of existing EEG signal classification models in practical applications and improves the generalization performance of the model.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, and in particular to a method for training an electroencephalogram (EEG) signal classification model, an EEG signal classification method, and an apparatus. Background Technology

[0002] Brain-computer interfaces (BCIs) are connections created between the human brain and external devices, enabling information exchange between them. BCI-based systems allow users to control devices to perform tasks using their brain signals. BCI systems have been widely applied in various fields. For example, in the gaming industry, BCI systems can control virtual characters on the game screen to perform actions such as walking, jumping, and turning based on the user's brain signals. After collecting the user's brain signals, determining the type of those signals to accurately analyze the corresponding control intentions is a crucial issue for BCI systems.

[0003] Currently, numerous machine learning methods are applied to the classification and analysis of electroencephalogram (EEG) signals. However, due to the significant differences in EEG signals among different users, classification models trained on data from a single user are difficult to apply to classifying the EEG signals of other users. This results in classification models with poor versatility in practical applications, making them unsuitable as universal classification models for multiple users. Summary of the Invention

[0004] This application provides a training method for an electroencephalogram (EEG) signal classification model, an EEG signal classification method, and an apparatus to address the problem of poor generalizability of existing EEG signal classification models in practical applications. By simultaneously training the model with source domain sample data and target domain sample data, the generalization performance of the model is improved.

[0005] Firstly, this application provides a method for training an electroencephalogram (EEG) signal classification model, including:

[0006] Acquire target domain sample data and source domain sample data; the target domain sample data includes the EEG signal of the target object, and the source domain sample data includes the EEG signal of auxiliary objects other than the target object;

[0007] Obtain an initial classification model to be trained, and perform preliminary training on the initial classification model based on the source domain sample data to obtain a candidate classification model that has been trained.

[0008] The candidate classification model is fine-tuned and trained based on the target domain sample data and the source domain sample data to obtain the trained target classification model; the target classification model is used to classify the EEG signals of the target object.

[0009] In the above technical solution, the model is initially trained using data information from the source domain to improve the prediction accuracy of the classification model. In order to avoid overfitting when fine-tuning the initially trained classification model using target domain sample data, the classification model is fine-tuned simultaneously using both source domain sample data and target domain sample data to obtain the trained target classification model, thereby improving the model's generalization ability.

[0010] Secondly, this application also provides a method for classifying electroencephalogram (EEG) signals, the method comprising:

[0011] Acquire the electroencephalogram (EEG) signals of the target subject;

[0012] The EEG signal is input into a pre-trained target classification method to obtain the classification result of the target object; the target classification model is a target classification model trained according to the EEG signal classification model training method described in the first aspect above.

[0013] In the above technical solution, the model is initially trained using data information from the source domain to improve the prediction accuracy of the classification model. To avoid overfitting when fine-tuning the initially trained classification model using target domain sample data, the classification model is fine-tuned simultaneously using both source domain sample data and target domain sample data to obtain a trained target classification model. This improves the model's generalization ability and thus enhances the processing efficiency and effectiveness when classifying the EEG signals of the target object.

[0014] Thirdly, this application also provides a training device for an electroencephalogram (EEG) signal classification model, comprising:

[0015] The sample data acquisition module is used to acquire target domain sample data and source domain sample data; the target domain sample data includes the electroencephalogram (EEG) signal of the target object, and the source domain sample data includes the EEG signal of auxiliary objects other than the target object.

[0016] The candidate classification model acquisition module is used to acquire an initial classification model to be trained, and to perform preliminary training on the initial classification model based on the source domain sample data to obtain a trained candidate classification model.

[0017] The target classification model acquisition module is used to fine-tune the candidate classification model based on the target domain sample data and the source domain sample data to obtain the trained target classification model; the target classification model is used to classify the EEG signals of the target object.

[0018] Optionally, the candidate classification model includes a feature extraction module, a first classification module, and a second classification module;

[0019] Correspondingly, the target classification model acquisition module includes:

[0020] The EEG feature acquisition submodule is used to input the target domain sample data and the source domain sample data into the feature extraction module in any iteration round to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain.

[0021] The classification prediction result acquisition submodule is used to input the target EEG features into the first classification module to obtain the target classification prediction result of the target domain, and to input the auxiliary EEG features into the second classification module to obtain the auxiliary classification prediction result of the source domain;

[0022] The model update submodule is used to generate a model loss function for the candidate classification model based on the target classification prediction result, the auxiliary classification prediction result, the target domain sample data, and the source domain sample data, and to update the model parameters of the candidate classification model based on the model loss function to obtain the candidate classification model after the current iteration.

[0023] Fourthly, this application also provides an electroencephalogram (EEG) signal classification device, the device comprising:

[0024] The EEG signal acquisition module is used to acquire the EEG signals of the target object;

[0025] The classification result acquisition module is used to input the EEG signal into a pre-trained target classification method to obtain the classification result of the target object; the target classification model is a target classification model trained according to the EEG signal classification model training method described in the first aspect above.

[0026] Fifthly, this application provides an electronic device, including: a processor, and a communication connection with the processor.

[0027] Memory;

[0028] The memory stores computer-executed instructions;

[0029] The processor executes computer execution instructions stored in the memory to implement the method as described in the first or second aspect.

[0030] In a sixth aspect, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first or second aspect.

[0031] In a seventh aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first or second aspect.

[0032] The model training method provided in this application uses data information from the source domain data for preliminary model training to improve the prediction accuracy of the classification model. In order to avoid overfitting when fine-tuning the preliminary training classification model with target domain sample data, the preliminary training classification model is fine-tuned simultaneously using both source domain sample data and target domain sample data to obtain the trained target classification model, thereby improving the generalization ability of the model. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0034] Figure 1 An application scenario diagram of the EEG signal classification model training method provided in this application;

[0035] Figure 2 A flowchart illustrating a brainwave signal classification model training method provided in this application embodiment;

[0036] Figure 3 A flowchart illustrating another method for training an EEG signal classification model provided in this application embodiment;

[0037] Figure 4 A structural diagram of the EEG signal classification model provided in this application during the initial training stage;

[0038] Figure 5 A structural diagram of the EEG signal classification model provided in this application during the fine-tuning training phase;

[0039] Figure 6 A schematic diagram showing the comparison of average accuracy of different training methods in the experimental description provided in this application;

[0040] Figure 7 A schematic diagram showing the distribution of prediction accuracy of different subjects using different Subject-adaptive algorithms in the experimental description provided for this application;

[0041] Figure 8 A schematic diagram showing the comparison of the average accuracy of different Subject-adaptive algorithms with different sample sizes in the experimental description provided for this application;

[0042] Figure 9 A flowchart illustrating an electroencephalogram (EEG) signal classification method provided in this application embodiment;

[0043] Figure 10 This is a schematic diagram of the structure of a brainwave signal classification model training device provided in this application according to an exemplary embodiment;

[0044] Figure 11 This is a schematic diagram of the structure of a brainwave signal classification model training device provided in this application according to an exemplary embodiment;

[0045] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0046] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0047] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0048] In practical applications, motor imagery brain-computer interfaces (BCIs) enable information exchange between the brain and external devices by decoding the electroencephalogram (EEG) signals generated during the brain's motor imagery process. Specifically, after acquiring the EEG signals, the BCI categorizes them to determine the corresponding control intentions, and then sends appropriate control commands to the devices. Therefore, determining the category of EEG signals to accurately analyze the corresponding control intentions is a crucial issue.

[0049] In practical applications, machine learning methods can be used to train neural networks to obtain classification models for determining the categories of EEG signals, thereby enabling subsequent control intentions in analyzing EEG signals. Specifically, machine learning studies how computers simulate human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve themselves. Therefore, in this embodiment, the classification model refers to predicting the category of new EEG signal data by learning from and training existing EEG signal data.

[0050] During training, sample data needs to be acquired in advance, and the model needs to be trained based on this data. During data collection, subjects must wear EEG caps throughout and perform a pre-defined motor imagery task. Therefore, data collection and annotation are very time-consuming. Furthermore, the sparse nature of data from individual subjects can lead to overfitting in classification models built based on labeled data from a single subject. In addition, differences in EEG signal frequencies among different subjects cause the data to not satisfy the independent and identically distributed hypothesis, resulting in classification models built based on labeled data from multiple subjects failing to achieve the desired prediction accuracy.

[0051] Based on this, the inventors of this application considered that by making full use of the data features contained in the EEG data of other subjects besides the target subject and the transfer learning model, and by training the classification model together with the EEG data of the other subjects after transfer and the EEG data of the target subject, the difference between the EEG signal frequencies of different subjects can be reduced, thereby improving the prediction accuracy of the classification model.

[0052] The research revealed that, building upon the aforementioned transfer learning, the Vanilla Fine-tuning algorithm employs a two-stage training strategy, including initial training and fine-tuning, which can improve the prediction accuracy of the classification model. Specifically, this method combines multiple subject data sets to form a source domain dataset for initial training of the classification model parameters. Subsequently, target subject data representing the target object is used to form a target domain dataset, and the initially trained classification model is fine-tuned using this target domain dataset to obtain a fully trained classification model. However, neural networks lack memory capabilities. When fine-tuning the classification model using the target domain dataset, it often forgets the knowledge learned from the source domain data, overfitting to the labeled target subject data, leading to reduced model generalization performance.

[0053] To address the aforementioned issues, Zhang et al. employed a strategy of early termination by dividing the target subjects into a validation set, thus mitigating the overfitting problem during the model fine-tuning phase. However, dividing the data into a validation set further reduces the amount of labeled data in the target domain, increasing the risk of overfitting during the model fine-tuning phase. Therefore, the inventors of this application are exploring how to determine the model training method without dividing the validation set to obtain a high-performance model, which is a crucial problem urgently needing to be solved in this field.

[0054] To address the aforementioned issues, the inventors discovered during their research that the classification model parameters initially trained in the two-stage training strategy retain prior knowledge contained in the source domain data. Existing training methods primarily mitigate overfitting by suppressing parameter updates during fine-tuning to preserve prior knowledge from the source domain data as much as possible. Optionally, Top-K Fine-tuning updates only the parameters of the last K layers of the network during fine-tuning, retaining the parameters of the shallow feature extraction module, thus preserving the potential information from the source domain data. Optionally, Weight Decay adds an L2 loss function (the difference between the model parameters and the pre-trained parameters) as a regularization term to suppress parameter updates during fine-tuning. Optionally, RecAdam, based on Weight Decay, constructs a weight term for the L2 loss that decays with the number of iterations, improving Weight Decay's fitting ability for downstream tasks. Optionally, Child-Tuning borrows the idea of ​​Dropout, updating only some parameters of the deep neural network during fine-tuning. During the implementation of the above method, it was found that while suppressing parameter updates can alleviate the catastrophic forgetting problem of Vanilla Fine-tuning in cases of sparse target domain samples, it sacrifices the model's ability to fit downstream tasks. Therefore, the inventors of this application are considering how to determine the model training method without sacrificing the model's ability to fit downstream tasks, so as to train a model with good performance. This is an important problem that urgently needs to be solved in this field.

[0055] Furthermore, the inventors of this application discovered during the research process that an important approach to solving the above problems is to introduce source domain data into the model's fine-tuning training phase to apply regularization, thereby mitigating the catastrophic forgetting problem and improving the model's generalization performance. Specifically, the source domain data consists of a combination of non-target subject data, performing the same motor imagery task. Although the source and target domains have different data distributions, they are very similar. Multi-task learning uses inductive transfer to utilize domain information from related tasks as inductive bias, improving the model's generalization performance. Therefore, the technical solution of this application considers drawing on the ideas of multi-task learning and proposes a training method for a multi-task fine-tuning strategy, namely MT-Tuning (Multi-Task Tuning). The multi-task aspect includes a pre-training task and a fine-tuning training task; specifically, in the fine-tuning task, the model needs to learn both source and target domain data simultaneously, mitigating the overfitting problem of the classification model in the target domain and improving the model's generalization performance.

[0056] Specifically, this application adopts the two-stage strategy of Vanilla Fine-tuning: pre-training and fine-tuning. In the pre-training stage, MT-Tuning initializes model parameters using source domain data and mines potential information from the source domain data. In the fine-tuning stage, MT-Tuning simultaneously uses both source and target domain data to fine-tune the model parameters. The source and target domain data share a feature extraction module but use different classification modules. During model training, in the forward prediction process, the source and target domain data are input into the shared feature extraction module. After learning their respective feature representations, they are passed through a gradient-weighted layer and then fed into their respective classification modules to obtain predicted labels. In the backpropagation process, the source and target domain loss functions update their respective classification modules, and then the weighted sum of the loss functions is calculated through a gradient-weighted layer to update the parameters of the shared feature extraction module. MT-Tuning has been further applied to a motor imagery EEG signal classification task, achieving cross-subject positive knowledge transfer. MT-Tuning outperformed deep neural network models trained using both subject-specific and subject-independent methods on the motion visualization task of the BMI dataset, achieving an average accuracy of 87.07%.

[0057] Figure 1 This diagram illustrates an application scenario of the EEG signal classification model training method provided in this application. For ease of understanding, the following diagram will be used in conjunction with... Figure 1 The application scenarios applicable to the embodiments of this application are described below. See also... Figure 1 The system pre-collects EEG signals from the target object as target domain sample data using a data acquisition device, and collects EEG signals from other auxiliary objects as source domain sample data. Based on this, in a training device, an initial classification model is initially trained using the source domain data to obtain a trained candidate classification model. Further, the candidate classification model is fine-tuned using both source and target domain sample data to obtain a trained target classification model. It should be noted that this target classification model is used for subsequent classification of the target object's EEG signals. The aforementioned pre-training using source domain data improves the prediction accuracy of the classification model. To avoid overfitting when fine-tuning the pre-trained classification model using target domain sample data, both source and target domain sample data are used simultaneously for fine-tuning to obtain the trained target classification model, thereby improving the model's generalization ability.

[0058] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0059] Figure 2 This is a flowchart illustrating a method for training an electroencephalogram (EEG) signal classification model, provided in an embodiment of this application. This method can be executed by an EEG signal classification model training device, which can be a server or an electronic device. The following description uses an electronic device as an example. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 2 As shown, the method includes the following steps:

[0060] S210. Obtain sample data from the target domain and sample data from the source domain.

[0061] In this application, the target domain sample data includes the electroencephalogram (EEG) signals of the target object, and the source domain sample data includes the EEG signals of auxiliary objects other than the target object. It should be noted that since both humans and animals can emit EEG signals, the target object and auxiliary object in this application can refer to either humans or animals; this embodiment does not specifically limit this. Because transfer learning is needed to transfer relevant characteristics of the source domain data to the target domain data, the transferred source domain data and target domain data jointly train the segmentation model to improve training performance. Therefore, the auxiliary object is any object other than the target object.

[0062] Specifically, the process of acquiring sample data can involve pre-assigning an imagery task to the subject, having the subject perform the task while wearing an EEG cap equipped with electrodes, and simultaneously acquiring the subject's EEG signals through a signal acquisition system connected to the EEG cap. These EEG signals, along with the corresponding imagery task, are then used as a set of sample data for subsequent model training. Optionally, this application can also employ other methods to acquire the EEG signals of the target subject and the auxiliary subject to obtain target domain sample data and sub-sample data. This application does not limit the method of acquiring sample data.

[0063] Specifically, the acquired sample data can be stored in a pre-defined sample database as a dataset. Specifically, the source domain sample data can be represented as: Among them, D s Let represent the source domain sample dataset, where s represents the source domain, x represents the EEG signal, y represents the signal label, n represents the number of signals, and i represents the i-th signal. The target domain sample data can be represented as: Among them, D tLet t represent the source domain sample dataset, x represent the EEG signal, y represent the signal label, n represent the number of signals, and i represent the i-th signal. Specifically, the target domain sample data includes both labeled and unlabeled data. The labeled data is as follows: Unlabeled data is

[0064] It should be noted that in this application, the source domain sample data and the target domain sample data follow data distributions P and Q, respectively, and P ≠ Q. The sample data size of the source domain sample data is much larger than the sample data size of the target domain sample data, i.e., n s >>n t =n l +n u .

[0065] S220. Obtain the initial classification model to be trained, and perform preliminary training on the initial classification model based on the source domain sample data to obtain the trained candidate classification model.

[0066] In this application, the initial segmentation model is pre-trained using source domain data to obtain a preliminary candidate segmentation model. This candidate segmentation model is then used as the basis for fine-tuning and training using target domain data and source domain data to obtain the final classification model.

[0067] In this embodiment, the initial classification model includes a feature extraction module and a classification module. Specifically, during iterative training, the EEG signals of the auxiliary object contained in the source domain sample data are input into the feature extraction module for EEG feature extraction, obtaining the auxiliary EEG features corresponding to the auxiliary object. These auxiliary EEG features are then input into the classification model for feature classification processing, resulting in a classification prediction result output by the classification module. Further, signal labels corresponding to the EEG signals of the auxiliary object in the source domain sample data are obtained, and the difference between the signal labels and the classification prediction result is calculated. This difference is used as the loss function of the initial classification model in the current iterative training process. Based on this loss function, the module parameters of the feature extraction module and the classification module are adjusted to obtain the updated classification model for the current iteration. Further, the updated classification model is used as the initial classification model for the next iteration training for the next round of iterative training until a preset iteration stopping condition is met, resulting in a trained candidate classification model. Optionally, the iteration stopping condition in this application can be that the number of iterations reaches a preset number, or that the generated loss function is less than a preset function threshold. Of course, other iteration stopping conditions can be set in advance, but this application does not impose specific limitations on them.

[0068] S230. Based on the target domain sample data and the source domain sample data, the candidate classification model is fine-tuned and trained to obtain the trained target classification model.

[0069] In this embodiment, the target classification model is used to classify the EEG signals of the target object. Therefore, after obtaining the initially trained candidate classification model, in order to obtain the final target classification model, it is necessary to fine-tune the candidate classification model using the EEG signals of the target object, i.e., target domain sample data. However, since neural networks do not have memory capabilities, using only target domain sample data for subsequent fine-tuning training can easily lead to overfitting of the classification model during the fine-tuning process, thereby reducing the predictive performance of the classification model. Therefore, in this application, to improve the predictive performance of the trained model, both source domain sample data and target domain sample data are used simultaneously to fine-tune the candidate classification model to obtain the final target classification model.

[0070] It should be noted that since two types of data are used for training during fine-tuning, the candidate classification model can include two feature extraction modules and two classification modules. Optionally, in some other embodiments, to simplify the model structure, a single feature extraction module can be used to extract EEG features from both the source domain sample data and the target domain sample data. However, due to individual differences, there are differences in signal distribution between different objects. Therefore, when fine-tuning the classification module, it is necessary to use sample data from a single object for training. That is, the first classification module is fine-tuned using the target domain sample data, and the second classification module is fine-tuned using the source domain sample data. Therefore, in the subsequent candidate classification model, two classification modules need to be pre-set. That is, the module parameters of the above classification modules are copied to obtain a first classification module for training the classification of EEG signals of the target object, and a second classification module for training the classification of EEG signals of the auxiliary object.

[0071] Optionally, the specific training process for fine-tuning the candidate classification model in this application will be described in detail in subsequent embodiments.

[0072] In the above technical solution, the model is initially trained using data information from the source domain to improve the prediction accuracy of the classification model. In order to avoid overfitting when fine-tuning the initially trained classification model using target domain sample data, the classification model is fine-tuned simultaneously using both source domain sample data and target domain sample data to obtain the trained target classification model, thereby improving the model's generalization ability.

[0073] Figure 3 This is a flowchart illustrating another method for training an EEG signal classification model provided in this application. This embodiment can be understood as a detailed description of the steps mentioned in the methods described above; see also... Figure 3The method may specifically include:

[0074] S310. Obtain sample data from the target domain and sample data from the source domain.

[0075] The target domain sample data includes the EEG signals of the target object, while the source domain sample data includes the EEG signals of auxiliary objects other than the target object.

[0076] Specifically, for the understanding and examples of the technical means, technical effects, and technical terms in step S310, please refer to the explanation of step S210 in the above embodiments.

[0077] S320. Obtain the initial classification model to be trained, and perform preliminary training on the initial classification model based on the source domain sample data to obtain the trained candidate classification model.

[0078] Specifically, for the understanding and examples of the technical means, technical effects, and technical terms in step S320, please refer to the explanation of step S220 in the above embodiments.

[0079] S330. Determine the pixels to be processed in the text image based on the channel reference pixel value, and perform pixel processing on the pixels to be processed to obtain the processed target text image.

[0080] Among them, the target classification model is used to classify the EEG signals of the target object.

[0081] Specifically, for the understanding and examples of the technical means, technical effects, and technical terms in step S330, please refer to the explanation of step S230 in the above embodiments.

[0082] Based on the above implementation, in this embodiment, the candidate classification model includes a feature extraction module, a first classification module, and a second classification module; correspondingly, in any iteration round, step S330 may specifically include:

[0083] S331. Input the target domain sample data and the source domain sample data into the feature extraction module respectively to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain.

[0084] To facilitate the model's better learning of the data information in the sample data, the candidate classification model in this application first inputs the sample data into the feature extraction module during training to extract features and obtain the corresponding EEG features. Specifically, the sample data in this application includes target domain sample data and source domain sample data, and correspondingly, the obtained EEG features also include target EEG features in the target domain and auxiliary EEG features in the source domain.

[0085] In this embodiment, the feature extraction module includes a shallow feature extraction layer, a downsampling layer, and a deep feature extraction layer. Correspondingly, the method of inputting target domain sample data and source domain sample data into the feature extraction module to obtain target EEG features of the target domain and auxiliary EEG features of the source domain may include: inputting target domain sample data and source domain sample data into the shallow feature extraction layer to obtain target shallow features of the target domain and auxiliary shallow features of the source domain; inputting target shallow features and auxiliary shallow features into the downsampling layer to obtain target downsampling features of the target domain and auxiliary downsampling features of the source domain; and inputting target downsampling features and auxiliary downsampling features into the deep feature extraction layer to obtain target EEG features of the target domain and auxiliary EEG features of the source domain.

[0086] Specifically, based on the signal characteristics of EEG signals, this application can extract sample features of sample data along the time and spatial dimensions when extracting shallow features. That is, feature extraction is performed on the aforementioned EEG signals to obtain the shallow features corresponding to the EEG signals. Optionally, the target EEG signal in the target domain is input into a pre-set one-dimensional temporal convolution kernel in the shallow feature extraction layer for temporal feature extraction to obtain the target EEG temporal features in the target domain; further, the target EEG temporal features are input into a pre-set one-dimensional spatial convolution kernel in the shallow feature extraction layer for spatial feature extraction to obtain the target shallow features in the target domain including spatial features. Optionally, based on the above method, auxiliary EEG signals in the source domain are input into a pre-set one-dimensional temporal convolution kernel and a one-dimensional spatial convolution kernel in the shallow feature extraction layer to obtain auxiliary shallow features in the source domain.

[0087] Furthermore, since the shallow features of the extracted EEG signals still have high dimensionality and are unsuitable for subsequent classification tasks, the technical solution of this application employs a one-dimensional average pooling strategy for feature downsampling. Specifically, the target shallow features obtained above are input into a downsampling layer for downsampling processing to obtain the target downsampled features in the target domain. Optionally, based on the above method, the auxiliary shallow features obtained above are also input into a downsampling layer for downsampling processing to obtain the auxiliary downsampled features in the source domain.

[0088] To further reduce the risk of overfitting during model training, the technical solution of this application performs deep feature extraction on the shallow features extracted above, facilitating the training of the subsequent classification module. Specifically, the target downsampled features obtained above are input into the deep feature extraction layer for deep feature extraction to obtain the target EEG features in the target domain; optionally, the auxiliary downsampled features obtained above are also input into the deep feature extraction layer for deep feature extraction based on the above method to obtain the auxiliary EEG features in the source domain.

[0089] S332. Input the target EEG features into the first classification module to obtain the target classification prediction result of the target domain, and input the auxiliary EEG features into the second classification module to obtain the auxiliary classification prediction result of the source domain.

[0090] In this embodiment of the application, in order to train the model’s classification learning ability, based on the obtained EEG features of the sample data, the EEG features are input into the classification module to obtain the classification prediction result.

[0091] In this application, due to individual differences, there are differences in signal distribution among different objects. Therefore, when fine-tuning the classification module, it is necessary to use sample data from a single object for training. That is, the first classification module is fine-tuned using target domain sample data, and the second classification model is fine-tuned using source domain sample data. Therefore, it is necessary to input the EEG features of different objects into different classification modules respectively.

[0092] Specifically, the target EEG features obtained in the target domain based on the above embodiments are input into the first classification module for classification prediction, and the target classification prediction result output by the first classification module is obtained. Optionally, based on the above embodiments, auxiliary EEG features in the source domain are also input into the second classification module for classification prediction, and the auxiliary classification prediction result output by the second classification module is obtained.

[0093] S333. Generate a model loss function for the candidate classification model based on the target classification prediction result, the auxiliary classification prediction result, the target domain sample data, and the source domain sample data. Update the model parameters of the candidate classification model based on the model loss function to obtain the candidate classification model after the current iteration.

[0094] In this embodiment, a loss function for the model is generated based on the classification prediction results and the signal labels corresponding to the EEG signals in the sample data. Then, the model parameters of the candidate classification model are adjusted based on the loss function during the fine-tuning training process to achieve the updating of the classification model during the fine-tuning training process.

[0095] Optionally, the candidate classification model in this application further includes a gradient weighting module. Correspondingly, the method for generating a model loss function for the candidate classification model based on the target classification prediction result, the auxiliary classification prediction result, the target domain sample data, and the source domain sample data, and updating the model parameters of the candidate classification model based on the model loss function to obtain the updated candidate classification model for the current iteration round may include: generating a first loss function based on the target classification prediction result and the target domain sample data, and adjusting the module parameters of the first classification module based on the first loss function to obtain the updated first classification module; generating a second loss function based on the auxiliary classification prediction result and the source domain sample data, and adjusting the module parameters of the second classification module based on the second loss function to obtain the updated second classification module; inputting the first and second loss functions into the gradient weighting module to obtain a third loss function, and adjusting the module parameters of the feature extraction module based on the third loss function to obtain the updated feature extraction module; and obtaining the updated candidate classification model for the current iteration round based on the updated first classification module, the updated second classification module, and the updated feature extraction module.

[0096] Specifically, based on the target classification prediction result, a first loss function is generated based on the target classification prediction result and the signal label corresponding to the target EEG signal in the target domain sample data. Then, the module parameters of the first classification module are adjusted based on the first loss function. Based on the auxiliary classification prediction result, a second loss function is generated based on the auxiliary classification prediction result and the signal label corresponding to the auxiliary EEG information signal in the source domain sample data. Then, the module parameters of the second classification module are adjusted based on the second loss function.

[0097] Furthermore, since the feature extraction module can simultaneously extract EEG features from both the target and source domains, it is necessary to adjust the parameters of the feature extraction module based on the loss function generated from the two sample data. Specifically, the first and second loss functions generated above are fused to obtain a fused loss function, which is then used to adjust the feature extraction module. Optionally, the first and second loss functions can be input into a gradient weighting module, where weights are assigned to the first and second loss functions, and a third loss function, fused based on their respective weights, is output. This third loss function is then used to adjust the feature extraction module. Further, the adjusted first classification module, second classification module, and feature module are combined to obtain the candidate classification model updated in the current round. Optionally, the candidate classification model is iteratively updated multiple times based on the above update method until a preset iteration stopping condition is met, at which point the update stops, and the candidate classification model after the update stops is used as the final target classification model. Optionally, the iteration stopping condition can be the same as or different from the stopping condition in the initial training phase; no specific limitation is made.

[0098] S334. Test the target classification model based on the target domain sample data, and determine the redundant processing module in the target classification model based on the test process of the model test.

[0099] In this embodiment, since sample data from multiple objects is required for model training during the training process, some processing modules need to be pre-set. For example, the processing modules may include a gradient weighting module and a second classification module corresponding to the source domain. However, the gradient weighting module and other object classification modules in the above processing modules are not used in the subsequent application of the model, i.e., in classifying EEG signals of the target object using the model. Therefore, if all processing modules and classification modules are retained in the target classification model during training and subsequent EEG signal classification is performed directly based on the target classification model, it may result in redundant processing modules in the target classification model, wasting storage space and processor resources, which will reduce the processing efficiency of the target classification model in EEG signal classification. Furthermore, since the processing flow of model prediction is consistent with the processing flow of subsequent model application for classification, this embodiment can process the redundant processing modules in the trained target classification model after training is completed.

[0100] Optionally, pre-set target test samples are obtained from the aforementioned target domain sample data, and these target test samples are input into the target classification model to obtain the target classification prediction result output by the model. Specifically, in the target classification model, the EEG signal of the target object is input into the feature extraction module to obtain the target EEG features output by the module, and then the target EEG features are input into the first classification module for feature classification to obtain the corresponding target classification test result. Based on the above process, it can be determined that two processing modules, the feature extraction module and the first classification module, are used in the target classification model during the testing process. Furthermore, based on the training process of the target classification model, it can be seen that the processing modules used in the target classification model during training are the feature extraction module, the gradient weighting module, the first classification module, and the second classification module. Therefore, based on the above fine-tuning training process and testing process, it can be known that the redundant processing modules in the target classification model include the gradient weighting module and the second classification module.

[0101] S335. Redundant processing modules in the target classification model are removed to obtain the processed target classification model.

[0102] In this embodiment, based on the above implementation method, the redundant modules are determined to include a gradient weighting module and a second classification module. The two modules are then deleted, and the connection relationships between other processing modules are adaptively modified so that the processed target classification model can still successfully classify EEG signals. That is, the feature extraction module is connected to the first classification module so that the features output by the feature extraction module can be directly used for subsequent classification processing.

[0103] In the above technical solution, the model is initially trained using data information from the source domain to improve the prediction accuracy of the classification model. To avoid overfitting when fine-tuning the initially trained classification model using target domain sample data, the classification model is simultaneously fine-tuned using both source domain and target domain sample data to obtain a trained target classification model, thereby improving the model's generalization ability. Furthermore, after training, the processing modules in the target classification model are simplified to obtain a processed target classification model, which reduces the storage space and resource utilization of the target classification model, thereby improving the classification efficiency of the target classification model in the subsequent EEG signal classification process.

[0104] Based on the above embodiments, this application also provides a preferred embodiment, which is another way of implementing the technical solution of this application. Figure 4 The structural diagram of the EEG signal classification model provided in this application during the initial training stage. Figure 5The diagram below shows the structure of the EEG signal classification model provided in this application during the fine-tuning training phase. For ease of understanding, it is illustrated below in conjunction with... Figures 4-5 The complete training process of the classification model in the embodiments of this application is described below. See [link to documentation]. Figure 4 The process involves obtaining an initial classification model to be trained, which includes a feature extraction model and a classification module. During the initial training of the initial classification model, since source domain data is used, source domain sample data is input into the initial classification model for iterative training, resulting in a preliminarily trained candidate classification model. It should be noted that because the initial training steps are the same as the subsequent fine-tuning training steps in this application, the specific training steps will only be described once; detailed training steps can be found in the subsequent fine-tuning training steps.

[0105] After fine-tuning the initial classification model, since the fine-tuning training phase uses two types of sample data, two classification modules are required. Specifically, the module parameters of the classification module are copied from the candidate classification model to obtain the first and second classification modules. Furthermore, to facilitate the backpropagation of the loss function to train the feature extraction module in subsequent training processes, a gradient weighting module is added to the candidate classification model. See [link to relevant documentation] Figure 5 The candidate classification model to be fine-tuned is obtained, which includes a feature extraction module, a gradient weighting module, a first classification module, and a second classification module.

[0106] Based on this, when fine-tuning the candidate classification model, the target sample data and source domain sample data are first obtained. Specifically, the above sample data can be expressed as x∈R C×T Correspondingly, the sample labels corresponding to the sample data can be expressed as y∈R Z Furthermore, the data have similar data distributions. Furthermore, for any of the above sample data, the sample data is input into the feature extraction module G(θ). g In the feature extraction module, the EEG features output by the module are obtained. Specifically, in the feature extraction module, two one-dimensional convolution operations, temporal convolution and spatial convolution, are used to extract features from the signals in both the temporal and spatial dimensions, respectively, to obtain the shallow features corresponding to the above sample data. It should be noted that in this application, the sample data can be first input into a one-dimensional temporal convolution kernel for extraction, and then the extracted features can be input into a one-dimensional spatial convolution kernel for feature extraction to obtain the corresponding shallow features; alternatively, the sample data can be simultaneously input into a one-dimensional temporal convolution kernel and a one-dimensional spatial convolution kernel, and then the obtained features can be processed to obtain the corresponding shallow features. This application does not limit the order of obtaining the shallow features.

[0107] Specifically, the input signal first undergoes temporal convolution to extract features, then spatial convolution to extract features, and then a batch normalization layer is used to normalize the extracted features. Finally, the nonlinear mapping features are obtained through the activation function f(·). Specifically, the expression (1) of the activation function is as follows:

[0108] H l =f(BN(W) l *X+b l )) (1)

[0109] Among them, W l and b l and represent the weights and biases of the l-th convolutional kernel, respectively. It should be noted that during the initial training phase, W... l and b l Random initialization; during the fine-tuning training phase, W l and b l We will continue using the parameter values ​​obtained during the initial training. * indicates a one-dimensional convolution operation, and BN indicates a batch normalization operation.

[0110] In this application, Exponential Linear Units (ELUs) can be selected as the nonlinear activation function f(·) for MT-Tuning. The advantage is that the ELU activation function, due to its near-zero output mean, avoids severe bias and accelerates network convergence. For example, the specific details of the ELU activation function are shown in the following expression (2):

[0111]

[0112] Optionally, the shallow features obtained after feature extraction still have high dimensionality and are not suitable for training the subsequent classification module. Therefore, it is necessary to downsample the features. Specifically, for any of the above sample data, the shallow features corresponding to the sample data are input into the downsampling layer, which can also be called the average pooling layer in this application. The shallow features are downsampled in this layer to obtain the processed downsampled features. For example, this application uses a one-dimensional average pooling strategy for downsampling. The specific details of the average pooling strategy are shown in the following expression (3):

[0113] H l =AvgPool(H l-1 (3)

[0114] Where Hl represents the processed downsampled features; Avgpool() represents the downsampling strategy; and Hl-1 represents the shallow features.

[0115] Optionally, to extract deeper features and reduce the risk of overfitting, the technical solution of this application also performs deep feature extraction on the downsampled features to obtain the final EEG features output by the model. In this application, depthwise separable convolution is used instead of traditional convolution operations for deep feature extraction. The reason is that depthwise separable convolution includes two parts: channel-wise convolution and pointwise convolution. Channel-wise convolution uses only one convolution kernel for each feature map, and pointwise convolution then obtains a new feature map by weighted summation of the channels. Therefore, while preserving the network's representational ability, the number of network parameters is reduced, thereby reducing the risk of overfitting.

[0116] Furthermore, based on the above method, the EEG signals corresponding to each sample data are obtained, and the obtained EEG signals are input into the corresponding classification modules for classification prediction. For example, for target domain sample data x... i t x is obtained based on the above method. i t The corresponding target EEG feature G(x) i t ;θ g Similarly, for source domain sample data x i s x is obtained based on the above method. i s Corresponding auxiliary EEG characteristics

[0117] Based on this, the first classification module is used as an example for illustrative purposes: the target EEG features in the target domain are input into the fully connected layer, the normalized exponential function and the prediction input layer in the first classification module to obtain the target classification prediction result output by the first classification module.

[0118] Specifically, the target EEG features G(x) obtained above are... i t ;θ g The input is fed into the first classification module, and the target classification prediction result y is obtained from the module output. i t The auxiliary EEG characteristics obtained above The input is fed into the second classification module, and the auxiliary classification prediction result y is obtained from the module's output. i sFor example, the expressions for the first classification module and the second classification module are shown in expressions (4) and (5) below:

[0119]

[0120]

[0121] Furthermore, during training, this application uses the cross-entropy loss function, commonly used in classification tasks, as the first loss function for adjusting the module parameters of the first classification module and the second loss function for adjusting the module parameters of the second classification module. For example, the first and second loss functions are shown in expressions (6) and (7) respectively:

[0122]

[0123]

[0124] in, Represents the first loss function; The second loss function is represented by z; the total number of categories is represented by c; and the corresponding category index is represented by n. s Indicates the number of source domain labeled data; n t Indicates the number of labels in the target domain; i represents the index of the labeled data. It is an indicator function; when hour, The value is 1 if it is not 1, and 0 otherwise. hour, The value is 1 if it is set to 1, otherwise the value is 0. This represents the network model, where θ is its corresponding parameter.

[0125] Furthermore, before updating the module parameters of the feature extraction module, a gradient weighting module is obtained. This module has pre-set weight parameters, including the weights corresponding to the first loss function and the second loss function. Optionally, these weight parameters can be manually specified, and they are not updated during backpropagation. The first loss function... Second loss function After passing through the gradient weighting layer, the two are weighted and summed to obtain the third loss function of the feature extraction module; the specific process of determining the third loss function is shown in the following expression (8):

[0126]

[0127] in, Let (1-λ) represent the third loss function; (1-λ) represent the weights corresponding to the first loss function; and λ represent the weights corresponding to the second loss function.

[0128] Furthermore, the adjusted first classification module, second classification module, and feature module are combined to obtain the candidate classification model updated in the current round. Optionally, the candidate classification model is iteratively updated multiple times based on the above update method until a preset iteration stopping condition is met, at which point the update stops, and the candidate classification model after the update stops is taken as the final target classification model.

[0129] After training, the processing modules in the target classification model are simplified to obtain a processed target classification model. This reduces the storage space and resource utilization of the target classification model, thereby improving the classification efficiency of the subsequent target classification model in the process of classifying EEG signals.

[0130] Building upon the above implementation, and after the model training is completed, to demonstrate the performance of the target classification model trained using the proposed training method, this application also describes the experiments conducted after training. Specifically, the following experimental procedures include an introduction to the experimental dataset and data preprocessing, an introduction to the experimental setup, an introduction to the comparative experiments, an introduction to the sample size experiment, an introduction to the model's real-time performance analysis, and an introduction to the experimental summary.

[0131] Specifically, Step 1: Introduction to Experimental Dataset and Data Preprocessing

[0132] This application uses the motor imagery portion of the BMI dataset published by the Department of Brain and Cognitive Engineering at Korea University as experimental data. The BMI dataset samples participants' EEG signals at a frequency of 1000 Hz using 62 electrodes. The dataset was collected from 54 participants (S1-S54) who performed left-hand and right-hand motor imagery tasks. Each participant's data was divided into two sessions, each containing 200 balanced samples. The first session data of the target participant was used as the target domain training set, and the second session data of the target participant was used as the target domain test set. All data from the remaining 53 non-target participants were combined to form the source domain data.

[0133] Deep learning models possess excellent feature extraction capabilities. To preserve as much input information as possible, data preprocessing is simplified to a minimum. For deep learning models, EEG signals from all electrodes in the dataset are retained. The EEG signals collected during the 4-second motor imagery task performed by the subject are extracted as input data and downsampled to 250Hz. Therefore, this is the input data for the deep learning model.

[0134] Step 2: Introduction to Experiment Setup

[0135] The proposed algorithm MT-Tuning is implemented using the PyTorch machine learning library. All experiments were run on the same device: an Intel(R) Xeon(R) Silver 4214 CPU and an Nvidia Tesla V100 GPU. The operating system was CentOS Linux release 7.9.2009, kernel version 3.10.0-1160.62.1.el7.x86_64. All deep neural networks in this application were trained using the mini-batch gradient descent algorithm. Except for experiments involving the number of samples, the batch size was uniformly set to 50. The optimizer was Adam, with corresponding parameters set to 0.9 and 0.99. For all models using the subject-independent training method, the learning rate and number of iterations were uniformly set to 0.01 and 200, respectively. For algorithms using the subject-adaptive training method, the learning rate and number of iterations remained consistent. During the pre-training phase, the learning rate and the number of iterations were set to 0.01 and 200, respectively; during the fine-tuning phase, the learning rate and the number of iterations were set to 0.003 and 300, respectively. This application uses accuracy and F-measure as evaluation metrics, where F-measure is the harmonic mean of precision and recall.

[0136] Step 3: Introduction to the Comparative Experiment

[0137] To verify the effectiveness of the proposed MT-Tuning, this application selected three classic neural network models for decoding EEG signals: EEGNet, ShallowConvNet, and DeepConvNet, as the network structure. EEGNet reduces the dimensionality of the input signal and decreases computational cost by downsampling the EEG signal to 250Hz. Simultaneously, EEGNet introduces depthwise separable convolutions instead of traditional convolutional layers, reducing computational cost and improving speed while maintaining network expressive power. Depthwise separable convolutions reduce the number of network parameters, mitigating the risk of overfitting on small datasets. ShallowConvNet mimics the feature extraction process of Filter Bank Common Spatial Pattern (FBCSP), using temporal and spatial convolutions to extract temporal and spatial information of the EEG signal, respectively. Subsequently, ShallowConvNet further downsamples and nonlinearly maps the features using squared activation functions, average pooling, and logarithmic activation functions. DeepConvNet uses four convolutional-pooling blocks to extract features layer by layer. The first convolutional-pooling block extracts original signal features through temporal and spatial convolutions, and max pooling downsamples the extracted features. Due to the low spatial resolution of EEG signals, the last three convolutional-pooling blocks in Deep ConvNet only reconstruct features through temporal convolutions, and max pooling downsamples the features. The ultimate goal of decoding EEG signals is to obtain a model with high prediction accuracy on the target subjects. However, EEG signal acquisition and annotation are expensive. Individual subjects often have only a few labeled samples, and data distribution is inconsistent among different subjects. Based on this characteristic of EEG signals, deep learning decoding algorithms mainly employ three training methods: Subject-specific, Subject-independent, and Subject-adaptive.

[0138] To ensure consistency in the distribution of training and testing data, the Subject-specific training method uses only the target subject's labeled data to train the model. To expand the training data scale, the Subject-independent training method ignores the differences in the distribution of different subject data and combines multiple subject data sets to form a training set. To fully explore the potential information of non-target subjects, the Subject-adaptive method achieves cross-subject knowledge transfer through pre-training and fine-tuning strategies. To verify the advantages of the proposed MT-Tuning algorithm compared to other training methods, the average accuracy of the three training methods is statistically analyzed, as shown in Table 1.

[0139] Table 1 Comparison of average accuracy of different training methods

[0140]

[0141] Figure 6 This is a schematic diagram comparing the average accuracy of different training methods in the experimental description provided in this application. For ease of understanding, it is combined with... Figure 6 The results of the comparative experiments used in this application are explained. For example... Figure 6 As shown, regardless of the network structure used, Subject-adaptive consistently achieves the highest prediction accuracy, Subject-independent performs slightly worse, and Subject-specific training results are significantly lower than the other two. Subject-specific training uses only the target subject's training samples to train model parameters; the BMI dataset contains only 200 labeled samples from the target subject. Subject-specific training suffers from severe overfitting issues and low prediction accuracy in such a limited sample size. The BMI dataset includes 54 different subjects, while the Subject-independent algorithm collects all non-target subject labeled data and test data, combining them with the target subject labeled data to form the training set. Therefore, in this experiment, the Subject-independent training set contains 21,200 labeled samples. Due to the large scale of the Subject-independent labeled data, the risk of overfitting is very small, and the model using Subject-independent training achieves high prediction performance. However, due to the inconsistent distribution of training and test data, Subject-independent prediction accuracy is limited. Subject-adaptive training pre-trains model parameters using all data from non-target subjects, then fine-tunes the parameters using labeled data from target subjects. The MT-Tuning training method proposed in this application introduces source domain data during the fine-tuning phase, applying regularization to the model based on the source domain data distribution, thus mitigating overfitting during the fine-tuning stage. MT-Tuning effectively addresses the shortcomings of subject-specific and subject-independent training methods, achieving optimal predictive performance.

[0142] Furthermore, the sample size of target subjects for motor imagery EEG signals is sparse, and the Subject-adaptive algorithm has a significant risk of overfitting. To verify the generalization performance of the MT-Tuning algorithm in this application, the embodiments of this application selected Vanilla Fine-tuning, Weight Decay, RecAdam, and Child-tuning as comparison algorithms. Among them, the effectiveness of Weight Decay, RecAdam, and Child-tuning has been verified in the field of natural language processing, and the embodiments of this application have transferred them to the field of EEG signal processing as comparison algorithms with MT-Tuning. Vanilla Fine-tuning does not use a regularization strategy in the fine-tuning stage, and fine-tunes the model parameters using training target domain data. Weight Decay adds an L2 regularization term between the current parameters of the model and the pre-trained parameters as a loss function to suppress parameter updates during fine-tuning. RecAdam, based on Weight Decay, constructs an L2 regularization term that continuously decays with the number of iterations, improving the model's ability to fit downstream tasks. Child-Tuning borrows the idea of ​​Dropout, updating only a portion of the neural network parameters during fine-tuning. The parameters of Weight Decay, RecAdam, and Child-Tuning were optimized. The average accuracy of different Subject-adaptive algorithms is shown in Table 2, and the distribution of Subject-adaptive algorithms across different subjects is as follows: Figure 7 As shown. {Pre-training, Vanilla Fine-tuning, Weight Decay, RecAdam, Child-Tuning, MT-Tuning} are abbreviated as {Pre-T, VF, WD, Rec, CT, MT-T} respectively.

[0143] See Figure 7 , Figure 7This paper compares the prediction performance of different Subject-adaptive algorithms under three network architectures. Vanilla Finetuning suffers from severe overfitting due to the lack of any regularization strategy. Under EEGNet, Shallow ConvNet, and Deep ConvNet network architectures, the average accuracy of Vanilla Finetuning decreased by 3.56%, 0.70%, and 3.48% respectively after fine-tuning. Weight Decay, RecAdam, and Child-Tuning alleviate the overfitting problem caused by the sparsity of the target subject labeled data during the fine-tuning stage by suppressing parameter updates, achieving slightly higher average prediction accuracy than Vanilla Finetuning. However, this regularization method of suppressing parameter updates sacrifices the model's ability to fit downstream tasks, resulting in lower prediction accuracy. The MT-Tuning model in this application introduces source domain data to jointly train the model parameters and applies regularization to the model's feature extraction module, thus mitigating the overfitting problem during the fine-tuning stage. The MT-Tuning model in this application achieved the highest prediction accuracy of 82.10%, 81.35%, and 87.07% in all three network structures, which is an improvement of 4.59%, 4.81%, and 6.36% respectively compared to the pre-trained model.

[0144] Table 2. Comparison of average accuracy of the Subject-adaptive algorithm

[0145]

[0146] Step 4: Sample Size Experiment Introduction

[0147] Acquiring and labeling motor imagery EEG signals is expensive. In real-world scenarios, target subjects may only have a very small amount of labeled data collected. To verify the predictive performance of the MT-Tuning model in scenarios with extremely limited sample sizes, this application embodiment designed a sample size experiment. Multiple subject-adaptive algorithms, including Vanilla Finetuning, Weight Decay, RecAdam, and Child-Tuning, were used as comparison algorithms to verify the predictive performance of the MT-Tuning model. In this application embodiment, N class-balanced labeled data were extracted from the target subject's training data. To ensure consistent parameter update frequency under different sample sizes, this application embodiment set the batch size to one-quarter of the sample size. To simplify the experimental process, the MT-Tuning parameter was fixed at 0.5, and the other subject-adaptive algorithms all selected the optimal parameters found in the comparison experiment. Specifically, the Weight Decay parameter was fixed at 0.1, the RecAdam parameters k and p were set to 0.2 and 200 respectively, and the Child-Tuning parameter p was set to 0.1. The embodiments of this application are based on the models EEGNet, Shallow ConvNet, and Deep ConvNet, respectively, and experiments are conducted. The average accuracy statistics of the Subject-Adaptive algorithm on the BMI dataset with different sample sizes are shown in Tables 3-5.

[0148] To more intuitively illustrate the accuracy distribution of different Subject-Adaptive algorithms on the BMI dataset, a line graph showing the accuracy changes of Subject-Adaptive algorithms with different sample sizes is presented, as shown below. Figure 8 As shown.

[0149] See details Figure 8The EEGNet pre-trained model achieved a prediction accuracy of 77.51%. However, due to the extremely sparse target sample size, Vanilla Finetuning encountered severe overfitting issues. With 200 samples, the average accuracy of Vanilla Finetuning was only 73.54%, a decrease of 3.97% compared to the pre-trained model. As the training data decreased, the risk of overfitting in Vanilla Finetuning gradually increased, leading to a decrease in model accuracy. With 8 samples, Vanilla Finetuning achieved an average accuracy of only 69.50%, a decrease of 8.01% after fine-tuning. Weight Decay, RecAdam, and Child-Tuning suppress parameter updates during the fine-tuning phase, applying regularization to the model and reducing the risk of overfitting during the fine-tuning process. When the network structure is EEGNet and the number of samples is 200, the accuracy is 74.95%, 74.05%, and 74.95%, respectively, which is slightly higher than the prediction accuracy of Vanilla Finetuning, but still lower than the average accuracy of 77.51% of the pre-trained model. While the strategy of suppressing parameter updates alleviates the overfitting problem during fine-tuning, it still cannot achieve positive knowledge transfer across subjects. With a sample size of 200, MT-Tuning achieves an average accuracy of 79.32, an increase of 1.79% compared to the pre-trained model, achieving positive knowledge transfer across subjects. As the number of samples decreases, the risk of overfitting in MT-Tuning increases, and the model's prediction accuracy gradually decreases. However, even in extremely sparse scenarios with only 16 and 8 samples, the MT-Tuning model in this application still maintains average accuracies of 77.45% and 77.64%, which are on par with the prediction results of the pre-trained model. Experimental results show that MT-Tuning introduces a feature extraction module trained on source domain data, thereby applying a regularization strategy to the module. This effectively alleviates the overfitting problem that occurs when fine-tuning a pre-trained large model in a small sample scenario and improves the model's generalization ability.

[0150] Table 3. Accuracy comparison of Subject-adaptive algorithm (EEGNet) with different sample sizes.

[0151]

[0152] Table 4. Accuracy comparison of Subject-adaptive algorithm (Shallow ConvNet) with different sample sizes.

[0153]

[0154] Table 5. Accuracy comparison of Subject-adaptive algorithm (Deep ConvNet) with different sample sizes.

[0155]

[0156] Step 5: Introduction to Model Real-Time Performance Analysis

[0157] The MT-Tuning algorithm in this application adopts a single-task classification approach. Under the condition of consistent network structure, the classification time is consistent with the Subject-specific and Subject-independent algorithms. The training process of MT-Tuning includes two stages: source domain pre-training and multi-task fine-tuning. This application's embodiments include a model real-time performance analysis to observe the difference in training time between the MT-Tuning model and the Subject-specific and Subject-independent algorithms. In this experiment, this application's embodiments selected the time required for a single iteration of different training algorithms as the comparison standard. The statistics of the time required for a single iteration of the three training methods (Subject-specific, Subject-independent, and Subject-adaptive) are shown in Table 6. Among them, the Subject-adaptive algorithm selected is the MT-Tuning model from this application.

[0158] Table 6 Comparison of training time (seconds) required per iteration for different training methods

[0159]

[0160] As shown in Table 6, the time required for a single iteration of the subject-specific training methods for network models EEGNet, Shallow ConvNet, and Deep ConvNet are 0.09 seconds, 0.17 seconds, and 0.15 seconds, respectively. With the same network model, the subject-specific training method requires the shortest training time. For network models EEGNet, Shallow ConvNet, and Deep ConvNet, the time required for a single iteration of the subject-independent training methods are 13.90 seconds, 30.83 seconds, and 22.37 seconds, respectively, significantly longer than the subject-specific training method. Subject-independent training involves combining all non-target subjects D... s and the labeled data of the target subjects D lThe training dataset is the largest in size, therefore requiring the longest time for a single iteration. The Subject-adaptive algorithm consists of two phases: pre-training and fine-tuning. The pre-training phase inputs non-target subject data D. s Initialize model parameters, and use the target subject's labeled data D during the fine-tuning phase. l Fine-tuning parameters. Since EEG signals from a single subject often have only a small number of labeled samples, while the source domain data obtained by combining multiple non-target subjects often has a large data volume, therefore T... pre >>T tuning The training time for the Subject-adaptive algorithm, including the pre-training and fine-tuning phases, is roughly equivalent to that of the Subject-independent algorithm. However, in practical scenarios, a training approach of one pre-training phase followed by multiple fine-tuning phases is typically used. Let T be the time required for pre-training. pre The time required for fine-tuning is T. tuning The fine-tuning training iterations are N. Each training iteration of Subject-independent training requires combining data from all non-target and target subjects, therefore the training time required for Subject-independent training is N(T). pre +T tuning Subject-adaptive training typically employs a pre-training phase followed by multiple fine-tuning phases, requiring only T seconds of training time. pre +NT tuning Compared to the Subject-independent training method, the Subject-adaptive training method can save (N-1)T. pre Therefore, Subject-adaptive training can significantly reduce training time in real-world scenarios, which is beneficial for EEG signal decoding research to overcome the limitations of hardware devices.

[0161] Step Six: Summary and Introduction of the Experiment

[0162] This application proposes MT-Tuning, a cross-subject motor imagery EEG signal classification algorithm based on a multi-task fine-tuning strategy. MT-Tuning trains model parameters using a two-stage strategy of pre-training and fine-tuning. In the pre-training stage, MT-Tuning uses source domain data and a single-task training approach to initialize model parameters and mine latent knowledge from the source domain data. In the fine-tuning stage, MT-Tuning simultaneously fits the source task and the target task, fine-tuning the model parameters. The source and target tasks share a feature extractor, and the output features are fed into their respective classifiers. By introducing the source task, regularization is applied to the feature extractor, mitigating the catastrophic forgetting problem in the fine-tuning stage; simultaneously, the target task uses a specific classifier, allowing the model to focus on fitting the target task and improving its fitting ability. MT-Tuning alleviates the catastrophic problem of Vanilla Finetuning in scenarios with sparse target domain samples, improving the model's generalization performance. Experiments on the BMI dataset show that MT-Tuning outperforms traditional EEG decoding algorithms and state-of-the-art deep learning algorithms, achieving an average accuracy of 87.07%. Furthermore, the sample size experiments demonstrate that MT-Tuning still performs well even with extremely sparse target domain samples. When the target domain sample size is only 32 and 16, MT-Tuning still achieves accuracies of 83.19% and 80.91%, respectively, surpassing the pre-trained model's prediction accuracy of 80.71%. In the network adaptive layer experiments, as the network's fitting ability increases, MT-Tuning exhibits good generalization performance, validating the effectiveness of MT-Tuning in mitigating overfitting.

[0163] MT-Tuning requires source domain data in both the pre-training and fine-tuning phases. Like other fine-tuning algorithms, its prediction accuracy is affected by the size of the source domain data. Furthermore, due to inconsistencies in the dimensions of signal data and labels acquired across devices and tasks, MT-Tuning cannot directly achieve cross-device and cross-task knowledge transfer. This will be a future research direction.

[0164] Figure 9 This is a flowchart illustrating a brainwave signal classification method provided in an embodiment of this application. The method can be executed by a brainwave signal classification device, which can be a server or an electronic device. The following description uses an electronic device as an example. The method in this embodiment can be implemented through software, hardware, or a combination of both. Figure 9 As shown, the method includes the following steps.

[0165] S410. Acquire the electroencephalogram (EEG) signals of the target object.

[0166] S420. Input the EEG signal into the pre-trained target classification method to obtain the classification result of the target object.

[0167] The target classification model is the target classification model trained according to the EEG signal classification model training method described in any embodiment.

[0168] In the above technical solution, the model is initially trained using data information from the source domain to improve the prediction accuracy of the classification model. To avoid overfitting when fine-tuning the initially trained classification model using target domain sample data, the classification model is fine-tuned simultaneously using both source domain sample data and target domain sample data to obtain a trained target classification model. This improves the model's generalization ability and thus enhances the processing efficiency and effectiveness when classifying the EEG signals of the target object.

[0169] Figure 10 This is a schematic diagram of the structure of a brainwave signal classification model training device provided in this application according to an exemplary embodiment. See also... Figure 10 The device includes: a sample data acquisition module 510, a candidate classification model acquisition module 520, and a target classification model acquisition module 530; wherein,

[0170] The sample data acquisition module 510 is used to acquire target domain sample data and source domain sample data; the target domain sample data includes the EEG signal of the target object, and the source domain sample data includes the EEG signal of auxiliary objects other than the target object.

[0171] The candidate classification model acquisition module 520 is used to acquire the initial classification model to be trained, perform preliminary training on the initial classification model based on the source domain sample data, and obtain the trained candidate classification model.

[0172] The target classification model acquisition module 530 is used to fine-tune the candidate classification model based on the target domain sample data and the source domain sample data to obtain the trained target classification model; the target classification model is used to classify the EEG signals of the target object.

[0173] Optionally, the candidate classification model includes a feature extraction module, a first classification module, and a second classification module;

[0174] Accordingly, the target classification model acquisition module 530 includes:

[0175] The EEG feature acquisition submodule is used to input the target domain sample data and the source domain sample data into the feature extraction module in any iteration round to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain.

[0176] The classification prediction result acquisition submodule is used to input the target EEG features into the first classification module to obtain the target classification prediction result of the target domain, and to input the auxiliary EEG features into the second classification module to obtain the auxiliary classification prediction result of the source domain;

[0177] The model update submodule is used to generate the model loss function of the candidate classification model based on the target classification prediction result, the auxiliary classification prediction result, the target domain sample data and the source domain sample data, and update the model parameters of the candidate classification model based on the model loss function to obtain the candidate classification model after the current iteration.

[0178] Optionally, the candidate classification model may also include a gradient weighting module;

[0179] Correspondingly, the model update submodule includes:

[0180] The first classification module update unit is used to generate a first loss function based on the target classification prediction result and the target domain sample data, and adjust the module parameters of the first classification module based on the first loss function to obtain the updated first classification module.

[0181] The second classification module update unit is used to generate a second loss function based on the auxiliary classification prediction results and source domain sample data, and adjust the module parameters of the second classification module based on the second loss function to obtain the updated second classification module.

[0182] The feature extraction module update unit is used to input the first loss function and the second loss function into the gradient weighting module to obtain the third loss function, and adjust the module parameters of the feature extraction module based on the third loss function to obtain the updated feature extraction module.

[0183] The candidate classification model update unit is used to obtain the updated candidate classification model for the current iteration based on the updated first classification module, the updated second classification module, and the updated feature extraction module.

[0184] Optionally, the feature extraction module includes a shallow feature extraction layer, a downsampling layer, and a deep feature extraction layer;

[0185] Correspondingly, the EEG feature acquisition submodule includes:

[0186] The shallow feature acquisition unit is used to input the target domain sample data and the source domain sample data into the shallow feature extraction layer respectively to obtain the target shallow features of the target domain and the auxiliary shallow features of the source domain.

[0187] The downsampling feature acquisition unit is used to input the target shallow features and auxiliary shallow features into the downsampling layer respectively to obtain the target downsampling features of the target domain and the auxiliary downsampling features of the source domain;

[0188] The EEG feature acquisition unit is used to input the target downsampled features and auxiliary downsampled features into the deep feature extraction layer to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain.

[0189] Optionally, after the candidate classification model stops training, the device further includes:

[0190] The redundancy processing module determination module is used to test the target classification model based on the target domain sample data, and to determine the redundancy processing module in the target classification model based on the test process of the model test.

[0191] The target classification model acquisition module is used to remove redundant processing modules in the target classification model to obtain the processed target classification model.

[0192] Figure 11 This is a schematic diagram of the structure of a brainwave signal classification model training device provided in this application according to an exemplary embodiment. See also... Figure 11 The device includes: an electroencephalogram (EEG) signal acquisition module 610 and a classification result acquisition module 620; wherein,

[0193] The EEG signal acquisition module 610 is used to acquire the EEG signals of the target object;

[0194] The classification result acquisition module 620 is used to input EEG signals into a pre-trained target classification method to obtain the classification result of the target object.

[0195] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 12 As shown, the electronic device in this embodiment may include:

[0196] At least one processor 701; and

[0197] Memory 702 communicatively connected to at least one processor;

[0198] The memory 702 stores instructions that can be executed by at least one processor 501, which, when executed by at least one processor 701, cause the server to perform the method as described in any of the above embodiments.

[0199] Alternatively, the memory 702 can be either standalone or integrated with the processor 701.

[0200] The implementation principle and technical effects of the electronic device provided in this embodiment can be found in the foregoing embodiments, and will not be repeated here.

[0201] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the method of any of the foregoing embodiments.

[0202] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the method of any of the foregoing embodiments.

[0203] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For instance, the division of modules is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0204] The integrated modules described above, implemented as software functional modules, can be stored in a computer-readable storage medium. These software functional modules, stored in a storage medium, include several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute some steps of the methods of the various embodiments of this application.

[0205] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. A general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor. The memory may include high-speed RAM, and may also include non-volatile memory (NVM), such as at least one disk storage device, and may also be a USB flash drive, external hard drive, read-only memory, disk, or optical disc, etc.

[0206] The aforementioned storage medium can be implemented from any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The storage medium can be any available medium accessible to general-purpose or special-purpose computers.

[0207] An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Alternatively, the storage medium can be an integral part of the processor. Both the processor and the storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and storage medium can exist as discrete components within a server or host device.

[0208] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0209] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for training an electroencephalogram (EEG) signal classification model, characterized in that, The method includes: Acquire target domain sample data and source domain sample data; the target domain sample data includes the EEG signal of the target object, and the source domain sample data includes the EEG signal of auxiliary objects other than the target object; Obtain an initial classification model to be trained, and perform preliminary training on the initial classification model based on the source domain sample data to obtain a candidate classification model that has been trained. The candidate classification model is fine-tuned and trained based on the target domain sample data and the source domain sample data to obtain the trained target classification model; the target classification model is used to classify the EEG signals of the target object. The candidate classification model includes a feature extraction module, a first classification module, and a second classification module; Accordingly, the candidate classification model is fine-tuned and trained based on the target domain sample data and the source domain sample data, including: In any iteration, the target domain sample data and the source domain sample data are input to the feature extraction module to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain; the target EEG features are input to the first classification module to obtain the target classification prediction result of the target domain, and the auxiliary EEG features are input to the second classification module to obtain the auxiliary classification prediction result of the source domain; the candidate classification model further includes a gradient weighting module; a first loss function is generated based on the target classification prediction result and the target domain sample data, and the module parameters of the first classification module are adjusted based on the first loss function. The updated first classification module is obtained; a second loss function is generated based on the auxiliary classification prediction result and the source domain sample data, and the module parameters of the second classification module are adjusted based on the second loss function to obtain the updated second classification module; the first loss function and the second loss function are input into the gradient weighting module to obtain a third loss function, and the module parameters of the feature extraction module are adjusted based on the third loss function to obtain the updated feature extraction module; based on the updated first classification module, the updated second classification module, and the updated feature extraction module, the candidate classification model updated in the current iteration is obtained.

2. The method according to claim 1, characterized in that, The feature extraction module includes a shallow feature extraction layer, a downsampling layer, and a deep feature extraction layer; Accordingly, the step of inputting the target domain sample data and the source domain sample data into the feature extraction module respectively to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain includes: The target domain sample data and the source domain sample data are respectively input into the shallow feature extraction layer to obtain the target shallow features of the target domain and the auxiliary shallow features of the source domain; The target shallow features and the auxiliary shallow features are respectively input into the downsampling layer to obtain the target downsampling features of the target domain and the auxiliary downsampling features of the source domain; The target downsampling feature and the auxiliary downsampling feature are respectively input into the deep feature extraction layer to obtain the target EEG feature of the target domain and the auxiliary EEG feature of the source domain.

3. The method according to claim 1, characterized in that, After the candidate classification model stops training, the method further includes: The target classification model is tested based on the target domain sample data, and the redundant processing modules in the target classification model are determined based on the test process of the model test. The redundant processing modules in the target classification model are removed to obtain the processed target classification model.

4. A method for classifying electroencephalogram (EEG) signals, characterized in that, The method includes: Acquire the electroencephalogram (EEG) signals of the target subject; The EEG signal is input into a pre-trained target classification method to obtain the classification result of the target object; the target classification model is a target classification model trained by the EEG signal classification model training method according to any one of claims 1 to 3.

5. A training device for a brainwave signal classification model, characterized in that, The device includes: The sample data acquisition module is used to acquire target domain sample data and source domain sample data; the target domain sample data includes the electroencephalogram (EEG) signal of the target object, and the source domain sample data includes the EEG signal of auxiliary objects other than the target object. The candidate classification model acquisition module is used to acquire an initial classification model to be trained, and to perform preliminary training on the initial classification model based on the source domain sample data to obtain a trained candidate classification model. The target classification model acquisition module is used to fine-tune and train the candidate classification model based on the target domain sample data and the source domain sample data to obtain the trained target classification model; the target classification model is used to classify the EEG signals of the target object; The candidate classification model includes a feature extraction module, a first classification module, and a second classification module; Accordingly, the target classification model acquisition module includes: The EEG feature acquisition submodule is used to input the target domain sample data and the source domain sample data into the feature extraction module in any iteration round to obtain the target EEG features of the target domain and the auxiliary EEG features of the source domain. The classification prediction result acquisition submodule is used to input the target EEG features into the first classification module to obtain the target classification prediction result of the target domain, and to input the auxiliary EEG features into the second classification module to obtain the auxiliary classification prediction result of the source domain; The model update submodule is used to generate a model loss function for the candidate classification model based on the target classification prediction result, the auxiliary classification prediction result, the target domain sample data, and the source domain sample data, and to update the model parameters of the candidate classification model based on the model loss function to obtain the candidate classification model after the current iteration. The candidate classification model also includes a gradient weighting module; Accordingly, the model update submodule includes: The first classification module update unit is used to generate a first loss function based on the target classification prediction result and the target domain sample data, and adjust the module parameters of the first classification module based on the first loss function to obtain the updated first classification module. The second classification module update unit is used to generate a second loss function based on the auxiliary classification prediction result and the source domain sample data, and adjust the module parameters of the second classification module based on the second loss function to obtain the updated second classification module. The feature extraction module update unit is used to input the first loss function and the second loss function into the gradient weighting module to obtain a third loss function, and adjust the module parameters of the feature extraction module based on the third loss function to obtain an updated feature extraction module; The candidate classification model update unit is used to obtain the candidate classification model updated in the current iteration based on the updated first classification module, the updated second classification module, and the updated feature extraction module.

6. A brainwave signal classification device, characterized in that, The device includes: The EEG signal acquisition module is used to acquire the EEG signals of the target object; The classification result acquisition module is used to input the EEG signal into a pre-trained target classification method to obtain the classification result of the target object; the target classification model is a target classification model trained by the EEG signal classification model training method according to any one of claims 1 to 3.

7. An electronic device, characterized in that, include: A processor and a memory communicatively connected to the processor; The memory stores computer-executed instructions; When executing the computer execution instructions, the processor is used to implement the EEG signal classification model training method as described in any one of claims 1 to 3 and the EEG signal classification method as described in claim 4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the EEG signal classification model training method as described in any one of claims 1 to 3 and the EEG signal classification method as described in claim 4.

Images