Method, device, equipment, medium and product for training electroencephalogram signal recognition model

Abstract

The application provides a brain electrical signal recognition model training method, device, equipment, medium and product, which can be used in the field of computer technology. The training method includes two stages of pre-training and fine-tuning. In the pre-training stage, the brain electrical signal recognition model is initialized based on the source domain sample data, so that the brain electrical signal recognition model has good recognition ability. In the fine-tuning stage, the input data of the brain electrical signal recognition model is a target synthetic electroencephalogram obtained by convex combination of the source domain electroencephalogram and the target domain electroencephalogram. The model parameters of the brain electrical signal recognition model are fine-tuned using the target synthetic electroencephalogram, so that the brain electrical signal recognition model can learn the correlation between the target synthetic electroencephalogram and the source domain label and the target domain label respectively. In this way, the problem of catastrophic forgetting in the iterative training process is alleviated, the generalization ability of the model is improved, and the accuracy of the model is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a training method, apparatus, device, medium and product for an electroencephalogram (EEG) signal recognition model. Background Technology

[0002] In the field of brain-computer interfaces, controlling external devices by decoding the brain's electroencephalogram (EEG) signals during motor imagery is an important research direction. However, due to the influence of factors such as muscle electrical signals and emotions, the data distribution varies among subjects, posing a challenge to the task of classifying EEG signals during motor imagery. Currently, the task of classifying EEG signals during motor imagery is generally achieved using EEG signal recognition models.

[0003] In related technologies, during the training process of EEG signal recognition models, the model is first pre-trained, and then the model parameters are fine-tuned. Vanilla Fine-tuning is a commonly used model fine-tuning method, widely applied in fields such as computer vision and natural language processing. When using Vanilla Fine-tuning to fine-tune EEG signal recognition models, the pre-trained model parameters retain the potential information from the source domain data, but catastrophic forgetting can occur during the fine-tuning process. Therefore, related technologies employ a method of suppressing parameter updates during the fine-tuning stage to mitigate this problem.

[0004] However, while suppressing parameter updates can alleviate the catastrophic forgetting problem of Vanilla Fine-tuning, it sacrifices the model's ability to fit downstream tasks. Therefore, how to mitigate the overfitting problem in the fine-tuning stage of Vanilla Fine-tuning while ensuring the model's ability to fit downstream tasks has become an urgent problem to be solved in this field. Summary of the Invention

[0005] This application provides a training method, apparatus, device, medium, and product for an electroencephalogram (EEG) signal recognition model, which can alleviate the catastrophic forgetting problem that may occur during iterative training, thereby improving the model's generalization ability and thus improving the model's accuracy.

[0006] In a first aspect, this application provides a training method for an electroencephalogram (EEG) signal recognition model, comprising:

[0007] The process involves acquiring source domain sample data and target domain sample data, and then calling a pre-trained EEG signal recognition model. The pre-trained EEG signal recognition model is obtained based on the source domain sample data, which includes multiple source domain EEG signals and corresponding source domain labels. The target domain sample data includes multiple target domain EEG signals and corresponding target domain labels. The source domain labels and target domain labels represent the motor imagery category to which the corresponding EEG signal belongs.

[0008] During the i-th iteration of fine-tuning the pre-trained EEG signal recognition model, samples are taken from the source domain sample data to obtain the source domain EEG signal and corresponding source domain label corresponding to the i-th iteration of training, and samples are taken from the target source sample data to obtain the target domain EEG signal and corresponding target domain label corresponding to the i-th iteration of training, where i is a positive integer greater than or equal to 1.

[0009] The source domain EEG signal and the target domain EEG signal are subjected to convex combination processing to obtain the target synthetic EEG signal.

[0010] Using the source domain label and the target domain label as supervision data, the pre-trained EEG signal recognition model is trained for the i-th iteration based on the target synthesized EEG signal to obtain the EEG signal recognition model after the i-th iteration training.

[0011] If it is determined that the EEG signal recognition model after the i-th iteration of training meets the preset convergence condition, then the EEG signal recognition model that meets the preset convergence condition is determined as the trained EEG signal recognition model. The trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

[0012] Secondly, this application provides a training device for an electroencephalogram (EEG) signal recognition model, comprising:

[0013] The acquisition module is used to acquire source domain sample data and target domain sample data, and call a pre-trained EEG signal recognition model; wherein, the pre-trained EEG signal recognition model is obtained by pre-training based on the source domain sample data, the source domain sample data includes multiple source domain EEG signals and source domain labels corresponding to each source domain EEG signal, and the target domain sample data includes multiple target domain EEG signals and target domain labels corresponding to each target domain EEG signal, wherein the source domain label and the target domain label respectively represent the motor imagery category to which the corresponding EEG signal belongs;

[0014] The sampling module is used to sample from the source domain sample data during the i-th iteration of fine-tuning the pre-trained EEG signal recognition model to obtain the source domain EEG signal and corresponding source domain label corresponding to the i-th iteration of training, and to sample from the target source sample data to obtain the target domain EEG signal and corresponding target domain label corresponding to the i-th iteration of training, where i is a positive integer greater than or equal to 1.

[0015] A convex combination processing module is used to perform convex combination processing on the source domain EEG signal and the target domain EEG signal to obtain a target synthetic EEG signal.

[0016] The training module is used to perform the i-th iteration training on the pre-trained EEG signal recognition model based on the target synthesized EEG signal using the source domain label and the target domain label as supervision data, so as to obtain the EEG signal recognition model after the i-th iteration training.

[0017] The determination module is used to determine the EEG signal recognition model that satisfies the preset convergence condition as the trained EEG signal recognition model if it is determined that the EEG signal recognition model after the i-th iteration of training satisfies the preset convergence condition. The trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

[0018] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor;

[0019] The memory stores computer-executed instructions;

[0020] The processor executes computer execution instructions stored in the memory to implement the training method for the EEG signal recognition model as described in any of the first aspects.

[0021] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the training method of the EEG signal recognition model according to any one of the first aspects.

[0022] Fifthly, this application provides a computer program product, including computer execution instructions, which, when executed by a processor, implement the training method for the EEG signal recognition model described in any of the first aspects.

[0023] This application provides a training method, apparatus, device, medium, and product for an EEG signal recognition model. The training method includes two stages: pre-training and fine-tuning. In the pre-training stage, the EEG signal recognition model is initialized based on source domain sample data, enabling it to possess good recognition capabilities. In the fine-tuning stage, the input data for the EEG signal recognition model is a target synthetic EEG signal obtained by convexly combining source domain EEG signals and target domain EEG signals. The model parameters are fine-tuned using the target synthetic EEG signal, allowing the model to learn the association relationships between the target synthetic EEG signal and the source domain labels and target domain labels, respectively. This mitigates the catastrophic forgetting problem that may occur during iterative training, thereby improving the model's generalization ability and ultimately its accuracy. Attached Figure Description

[0024] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0025] Figure 1 This is a schematic diagram illustrating an application scenario according to an exemplary embodiment;

[0026] Figure 2 This is a flowchart illustrating a training method for an electroencephalogram (EEG) signal recognition model according to an exemplary embodiment.

[0027] Figure 3 This is a flowchart illustrating a training method for an electroencephalogram (EEG) signal recognition model according to another exemplary embodiment;

[0028] Figure 4 This is a schematic diagram illustrating a fine-tuning stage according to an exemplary embodiment;

[0029] Figure 5 This is a schematic diagram of a line graph according to an exemplary embodiment;

[0030] Figure 6 This is a schematic diagram of the structure of a training device for an electroencephalogram (EEG) signal recognition model according to an exemplary embodiment;

[0031] Figure 7 This is a schematic diagram of the structure of an electronic device according to an exemplary embodiment.

[0032] The accompanying drawings illustrate specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0033] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0034] The terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. In the following descriptions of embodiments, "a plurality of" means two or more, unless otherwise explicitly defined.

[0035] To clearly understand the technical solution of this application, the solutions of the prior art will be described in detail first.

[0036] In the field of brain-computer interfaces, controlling external devices by decoding the electroencephalogram (EEG) signals generated during motor imagery is an important research direction. However, due to the influence of factors such as electromuscular signals and emotions, the data distribution varies among subjects, posing a challenge to the task of classifying EEG signals during motor imagery. Currently, the task of classifying EEG signals during motor imagery is generally achieved using EEG signal recognition models.

[0037] In related technologies, during the training process of EEG signal recognition models, the model is first pre-trained, and then the model parameters are fine-tuned. Vanilla fine-tuning is a commonly used model fine-tuning method, widely applied in fields such as computer vision and natural language processing. However, due to the high cost of EEG signal data acquisition and annotation, a single subject can only provide a small number of labeled samples. When using Vanilla fine-tuning to fine-tune EEG signal recognition models, the pre-trained model parameters retain the potential information from the source domain data, but catastrophic forgetting occurs during the fine-tuning process. Therefore, related technologies employ a method of suppressing parameter updates during the fine-tuning stage to mitigate this problem.

[0038] Among them, the Top-K method only fine-tunes the parameters of the last K layers of the network, which can reduce the number of adjustable parameters and retain the information of the source domain data; the Weight Decay method uses the L2 loss function, which is the difference between the model parameters and the pre-trained parameters, as a regularization term to suppress parameter updates during the fine-tuning stage; the RecAdam method builds a weight term that continuously decays with the number of iterations based on the Weight Decay method, which is used for training the L2 loss to improve the Fitting Ability of the Weight Decay Method to Downstream Tasks; the Child-Tuning method only updates a portion of the parameters of the deep neural network, similar to the idea of ​​Dropout (random deactivation).

[0039] While the above-mentioned method of inhibiting parameter updates can alleviate the catastrophic forgetting problem of Vanilla Fine-tuning, EEG signal recognition models often suffer from overfitting, resulting in insufficient generalization ability and low accuracy.

[0040] To address the overfitting problem often found in existing EEG signal recognition models, leading to insufficient generalization ability and accuracy, the inventors discovered that, to solve this problem, source domain data can be introduced during the fine-tuning stage of the EEG signal recognition model. This allows the model to learn not only target domain data but also source domain data, thus mitigating catastrophic forgetting and reducing overfitting, thereby improving generalization ability and accuracy. Specifically, the training method for the EEG signal recognition model provided in this application includes two stages: pre-training and fine-tuning. In the pre-training stage, the EEG signal recognition model is initialized based on source domain sample data, enabling it to possess better recognition capabilities. In the fine-tuning phase, the input data for the EEG signal recognition model is the target synthetic EEG signal obtained by convexly combining the source domain EEG signal and the target domain EEG signal. The model parameters of the EEG signal recognition model are fine-tuned using the target synthetic EEG signal, so that the EEG signal recognition model can learn the correlation between the target synthetic EEG signal and the source domain label and the target domain label, respectively. This can alleviate the catastrophic forgetting problem that may occur during iterative training, thereby improving the model's generalization ability and thus improving the model's accuracy.

[0041] The following describes the application scenarios of the training method for the EEG signal recognition model provided in the embodiments of this application.

[0042] Figure 1 This is a schematic diagram illustrating an application scenario according to an exemplary embodiment. For example... Figure 1As shown, this application scenario includes an electronic device 1. The electronic device 1 is used to train an EEG signal recognition model. In this embodiment, during the iterative training of the pre-trained EEG signal model in the fine-tuning stage, the electronic device samples source domain EEG signals and corresponding source domain labels from source domain sample data, and samples target domain EEG signals and corresponding target domain labels from target domain sample data. Then, it performs convex combination processing on the source domain and target domain EEG signals to obtain a target synthetic EEG signal. Using the source domain labels and target domain labels as supervision data, iteratively trains the pre-trained EEG signal recognition model based on the target synthetic EEG signal. The EEG signal recognition model that meets the preset convergence condition is determined as the successfully trained EEG signal recognition model.

[0043] The technical solution of this application and how it solves the above-mentioned technical problems will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0044] Figure 2 This is a flowchart illustrating a training method for an electroencephalogram (EEG) signal recognition model according to an exemplary embodiment. The execution entity of the EEG signal recognition model training method provided in this application is an EEG signal recognition model training device, which is integrated into an electronic device. For example... Figure 2 As shown, the training method for the EEG signal recognition model provided in this embodiment includes the following steps:

[0045] Step S101: Obtain source domain sample data and target domain sample data, and call the pre-trained EEG signal recognition model.

[0046] Among them, the pre-trained EEG signal recognition model is obtained by pre-training based on source domain sample data. The source domain sample data includes multiple source domain EEG signals and source domain labels corresponding to each source domain EEG signal. The target domain sample data includes multiple target domain EEG signals and target domain labels corresponding to each target domain EEG signal. The source domain label and the target domain label respectively represent the motor imagery category to which the corresponding EEG signal belongs.

[0047] In this embodiment, both the source domain sample data and the target domain sample data are pre-collected, and the source domain labels corresponding to the source domain EEG signals and the target domain labels corresponding to the target domain EEG signals are manually annotated. The source domain labels and target domain labels respectively represent the motor imagery category to which the corresponding EEG signal belongs. The number of motor imagery categories is at least two, and this number can be set as needed; this embodiment does not limit this. For example, the number of motor imagery categories is two: left-hand motor imagery and right-hand motor imagery. Alternatively, the number of motor imagery categories is four: left-hand motor imagery, right-hand motor imagery, left-foot motor imagery, and right-foot motor imagery.

[0048] In this embodiment, the pre-trained EEG signal recognition model is trained to fine-tune the model parameters. That is, the training process of the EEG signal model includes two stages: a pre-training stage and a fine-tuning stage.

[0049] Step S102: During the i-th iteration of fine-tuning the pre-trained EEG signal recognition model, samples are taken from the source domain sample data to obtain the source domain EEG signal and corresponding source domain label corresponding to the i-th iteration of training, and samples are taken from the target source sample data to obtain the target domain EEG signal and corresponding target domain label corresponding to the i-th iteration of training, where i is a positive integer greater than or equal to 1.

[0050] The training process for the pre-trained EEG signal recognition model includes multiple iterative training sessions, and the number of iterations can be set as needed; this embodiment does not limit this. Steps S102-S105 are explained using the i-th iteration as an example, where the i-th iteration is any one of the multiple iterations. During the i-th iteration, samples are taken from both the source domain and target domain sample data to obtain the source domain EEG signal and its corresponding source domain label, and the target domain EEG signal and its corresponding target domain label corresponding to the i-th iteration. The sampling method can be set as needed; this embodiment does not limit this, for example, random sampling. Optionally, the number of source domain EEG signals and target domain EEG signals obtained from the sampling is one each.

[0051] Step S103: Perform convex combination processing on the source domain EEG signal and the target domain EEG signal to obtain the target synthetic EEG signal.

[0052] In this step, a target synthetic EEG signal is obtained by performing convex combination processing on the source domain EEG signal and the target domain EEG signal. This target synthetic EEG signal contains information from both the source domain EEG signal and the target domain EEG signal, and therefore contains relatively rich information.

[0053] Step S104: Using source domain labels and target domain labels as supervision data, the pre-trained EEG signal recognition model is trained for the i-th iteration based on the target synthesized EEG signal to obtain the EEG signal recognition model after the i-th iteration training.

[0054] The pre-trained EEG signal recognition model takes the target synthetic EEG signal as input and outputs at least two predicted probabilities corresponding to the target synthetic EEG signal. Each predicted probability represents the probability that the source domain EEG signal belongs to the corresponding motor imagery category. The EEG signal recognition model determines the probability that the input target synthetic EEG signal belongs to each motor imagery category, thereby obtaining the predicted probability corresponding to each motor imagery category.

[0055] Step S105: If it is determined that the EEG signal recognition model after the i-th iteration of training meets the preset convergence condition, then the EEG signal recognition model that meets the preset convergence condition is determined as the trained EEG signal recognition model. The trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

[0056] In this embodiment, after each iteration of training, it is determined whether the EEG signal recognition model meets a preset convergence condition. EEG signal recognition models that meet the preset convergence condition are then identified as successfully trained EEG signal recognition models, completing the iterative training of the fine-tuning phase. The preset convergence condition can be set as needed, and this embodiment does not limit it. For example, the preset convergence condition is that the model loss value is less than a preset loss value. Alternatively, the preset convergence condition is that the number of iterations reaches a preset number.

[0057] The training method for the EEG signal recognition model provided in this embodiment includes two stages: pre-training and fine-tuning. In the pre-training stage, the EEG signal recognition model is initialized based on source domain sample data, enabling it to possess good recognition capabilities. In the fine-tuning stage, the input data for the EEG signal recognition model is a target synthetic EEG signal obtained by convexly combining source domain EEG signals and target domain EEG signals. The model parameters are fine-tuned using the target synthetic EEG signal, allowing the model to learn the association relationships between the target synthetic EEG signal and the source domain labels and the target domain labels, respectively. This mitigates the catastrophic forgetting problem that may occur during iterative training, thereby improving the model's generalization ability and ultimately its accuracy.

[0058] In the above Figure 2Based on the illustrated embodiment, before step S101, the electronic device pre-trains the EEG signal recognition model to obtain a pre-trained EEG signal recognition model. In an optional embodiment, to fully explore the potential information in the source domain sample data and improve the generalization performance of the model, source domain sample data can be used for pre-training. Accordingly, the pre-training process includes: using the source domain labels corresponding to each source domain EEG signal as supervision data, pre-training the EEG signal recognition model based on each source domain EEG signal to obtain a pre-trained EEG signal recognition model. The input data for the EEG signal recognition model during the pre-training stage is the source domain EEG signal.

[0059] Optionally, during the i-th iteration of the pre-training phase of the EEG signal recognition model, samples are taken from the source domain sample data to obtain the source domain EEG signal and its corresponding source domain label corresponding to the i-th iteration of training, where i is a positive integer greater than or equal to 1; the source domain EEG signal is input into the EEG signal recognition model, and the EEG signal recognition model is used to determine at least two predicted probabilities corresponding to the source domain EEG signal, each predicted probability representing the probability that the source domain EEG signal belongs to the corresponding motor imagery category; based on the at least two predicted probabilities corresponding to the source domain EEG signal and the source domain label, the i-th iteration of training is determined. The pre-training loss value corresponding to the i-th iteration training is used; based on the pre-training loss value corresponding to the i-th iteration training, the model parameters of the EEG signal recognition model are adjusted to obtain the EEG signal recognition model after the i-th iteration training; if the EEG signal recognition model after the i-th iteration training meets the preset convergence condition, then the EEG signal recognition model that meets the preset convergence condition is determined as the pre-trained EEG signal recognition model; if the EEG signal recognition model after the i-th iteration training does not meet the preset convergence condition, then the EEG signal recognition model is trained for the (i+1)-th iteration.

[0060] The pre-training phase includes multiple iterations of training, and the number of iterations can be set as needed; this embodiment does not limit this. The i-th iteration is any one of the multiple iterations. During the i-th iteration, samples are taken from the source domain sample data to obtain the source domain EEG signal and corresponding source domain label corresponding to the i-th iteration. The sampling method can be set as needed; this embodiment does not limit this, for example, random sampling. Optionally, the number of source domain EEG signals obtained is one. The preset convergence condition can be set as needed; this embodiment does not limit this, for example, the preset convergence condition can be the same as or different from the preset convergence condition in the fine-tuning phase.

[0061] Optionally, the process of determining the pre-training loss value corresponding to the i-th iteration training based on at least two predicted probabilities corresponding to the source domain EEG signal and the source domain label is implemented by the loss function in the pre-training stage, and the loss function in the pre-training stage is shown in Formula 1 below:

[0062] Formula 1:

[0063] in, Let i ∈ [1, n] be the loss function during the pre-training phase. s ], n s c represents the number of source domain EEG signals included in the source domain sample data, c∈[1,z], where z represents the number of motor imagery categories. It is an indicator function. Indicates when hour, The value is 1, otherwise The value is 0, and log represents the logarithmic function. Represents source domain EEG signals, This indicates the source domain label corresponding to the source domain EEG signal. This represents the EEG signal recognition model, where θ represents the model parameters. This represents the predicted probability.

[0064] The above optional embodiments describe the pre-training stage of the EEG signal recognition model. The fine-tuning stage of the EEG signal recognition model will be described below.

[0065] Figure 3 This is a flowchart illustrating a training method for an electroencephalogram (EEG) signal recognition model according to another exemplary embodiment, such as... Figure 3 As shown, the training method for the EEG signal recognition model provided in this embodiment is... Figure 2 Based on the training method of the EEG signal recognition model provided in the illustrated embodiment, steps S103 and S104 are further refined. The training method of the EEG signal recognition model provided in this embodiment includes the following steps:

[0066] Step S201: Obtain source domain sample data and target domain sample data.

[0067] The source domain sample data includes multiple source domain EEG signals and source domain labels corresponding to each source domain EEG signal. The target domain sample data includes multiple target domain EEG signals and target domain labels corresponding to each target domain EEG signal. The source domain labels and target domain labels represent the motor imagery category to which the corresponding EEG signals belong.

[0068] Among them, the source domain sample data is Target domain sample data Represents a source domain sample. Represents source domain EEG signals, This represents the source domain label corresponding to the source domain EEG signal, i∈[1,n]. s ], ns This indicates the number of source domain EEG signals included in the source domain sample data. Represents a target domain sample. Represents the target domain EEG signal. This represents the target domain label corresponding to the target domain EEG signal, j∈[1,n]. l ], n l This indicates the number of target domain EEG signals included in the target domain sample data.

[0069] It should be noted that the source domain sample data includes a larger number of source domain EEG signals, while the target domain sample data includes a smaller number of target domain EEG signals. s >>n l The source domain sample data and the target domain sample data share the same input space. and tag space They follow different data distributions, but the distributions among the data are very similar. Let z represent a real number, and z represent the number of categories of motion imagination.

[0070] Step S202: Call the pre-trained EEG signal recognition model.

[0071] The pre-trained EEG signal recognition model is obtained by pre-training based on source domain sample data.

[0072] In this embodiment, the pre-training process of the EEG signal recognition model is the same as that in the above embodiment, and the implementation of steps S201-S202 is the same as that of step S101, so it will not be described again here.

[0073] After step S202, the pre-trained EEG signal recognition model is subjected to iterative training in the fine-tuning stage. The following description takes the i-th iteration of the fine-tuning stage as an example. The i-th iteration includes the following steps S203-S207, where i is a positive integer greater than or equal to 1.

[0074] Step S203: Sample source domain EEG signals and corresponding source domain labels from source domain sample data, and sample target domain EEG signals and corresponding target domain labels from target source sample data.

[0075] In this embodiment, the implementation of step S203 is the same as that of step S102, and will not be described again here.

[0076] Step S204: Perform convex combination processing on the source domain EEG signal and the target domain EEG signal to obtain the target synthetic EEG signal.

[0077] Optionally, this embodiment uses an inter-domain mixup method to generate the target synthetic EEG signal. Accordingly, the implementation of step S204 includes: obtaining multiple inter-domain interpolation coefficients that follow a beta distribution; using each inter-domain interpolation coefficient as a weight, performing weighted summation on the corresponding source domain EEG signal and the corresponding target domain EEG signal to obtain the neighborhood distribution corresponding to the source domain EEG signal and the target domain EEG signal, the neighborhood distribution including multiple synthetic EEG signals; and randomly sampling from the neighborhood distribution to obtain the target synthetic EEG signal.

[0078] The number of target synthesized EEG signals is at least one, and this number can be set as needed; this embodiment does not limit it. For example, a dataset can be obtained by randomly sampling from a neighborhood distribution. This dataset includes m target synthetic EEG signals. Synthesize an EEG signal for the k-th target, where k∈[1,m]. This represents the source domain label corresponding to the synthesized EEG signal of the k-th target. This source domain label is the source domain label corresponding to the source domain EEG signal. This represents the target domain label corresponding to the k-th target synthesized EEG signal. This target domain label is the target domain label corresponding to the target domain EEG signal. During the fine-tuning phase, this dataset is considered as input data for the EEG signal recognition model.

[0079] In this embodiment, inter-domain interpolation coefficients are first used to perform weighted summation on the source domain EEG signal and the target domain EEG signal, thereby constructing the neighborhood distribution of the source domain EEG signal and the target domain EEG signal. Then, random sampling is performed on the neighborhood distribution to obtain the target synthetic EEG signal. This realizes the construction of the input data for the EEG signal recognition model in the fine-tuning stage. The constructed input data contains information from both the source domain EEG signal and the target domain EEG signal, thus containing relatively rich information and high-quality input data.

[0080] Based on the above optional implementation methods, for example, the step of using the inter-domain interpolation coefficients as weights to perform weighted summation on the corresponding source domain EEG signals and the corresponding target domain EEG signals to obtain the neighborhood distributions of the source domain EEG signals and the target domain EEG signals is implemented by the following formula two:

[0081] Formula 2:

[0082] Among them, P v Indicates neighborhood distribution, Indicates synthesized EEG signals, Represents the target domain EEG signal. Represents the source domain EEG signal, i∈[1,n] s ],j∈[1,n l], n s n represents the number of source domain EEG signals included in the source domain sample data. l This represents the number of target domain EEG signals included in the target domain sample data, where λ is the inter-domain data interpolation coefficient, λ ~ Beta(α,α), α∈(0,+∞]. Let denot represent the values ​​of all λ, and δ represent the Dirac function.

[0083] This embodiment constructs the neighborhood distribution of the source domain EEG signal and the target domain EEG signal by convex combination, thereby providing data support for the construction of input data.

[0084] Following step S204, using source domain labels and target domain labels as supervised data, the pre-trained EEG signal recognition model is trained for the i-th iteration based on the target synthesized EEG signal to obtain the EEG signal recognition model after the i-th iteration training. Optionally, this process is implemented by the following steps S205-S207.

[0085] Step S205: Input the target synthesized EEG signal into the pre-trained EEG signal recognition model, and use the pre-trained EEG signal recognition model to determine at least two predicted probabilities corresponding to the target synthesized EEG signal.

[0086] The pre-trained EEG signal recognition model takes as input the target synthetic EEG signal and outputs at least two predicted probabilities corresponding to the target synthetic EEG signal. Each predicted probability represents the probability that the target synthetic EEG signal belongs to the corresponding motor imagery category. The EEG signal recognition model determines the probability that the input target synthetic EEG signal belongs to each motor imagery category, thereby obtaining the predicted probability corresponding to each motor imagery category.

[0087] Step S206: Based on at least two predicted probabilities, source domain labels, and target domain labels corresponding to the target synthesized EEG signal, determine the model loss value corresponding to the i-th iteration training.

[0088] For example, step S206 is implemented by: determining the motion imagery category corresponding to the largest prediction probability among at least two prediction probabilities as the predicted motion imagery category; using a cross-entropy function to determine the source domain loss value between the predicted motion imagery category and the source domain label, and using a cross-entropy function to determine the target domain loss value between the predicted motion imagery category and the target domain label; and performing a weighted summation of the source domain loss value and the target domain loss value to obtain the model loss value corresponding to the i-th iteration of training.

[0089] The higher the prediction probability, the more likely the synthesized EEG signal belongs to the corresponding motor imagery category; conversely, the lower the prediction probability, the less likely the synthesized EEG signal belongs to the corresponding motor imagery category. Therefore, the motor imagery category corresponding to the highest prediction probability can be determined as the predicted motor imagery category. To meet the requirements of the cross-entropy loss function, the source domain label and target domain label are represented in one-hot code form.

[0090] In this embodiment, source domain labels and target domain labels are used as supervision data, and the cross-entropy loss function is used to determine the source domain loss value and the target domain loss value, respectively, so as to obtain the model loss value corresponding to the i-th iteration training. The determined model loss value is relatively accurate.

[0091] For example, step S206 is implemented using the loss function of the fine-tuning stage, and the loss function of the fine-tuning stage is shown in Formula 3 below:

[0092] Formula 3:

[0093] in, Let λ represent the loss function during the fine-tuning phase, k∈[1,m], m represent the number of target synthesized EEG signals, λ be the inter-domain data interpolation coefficients, and c∈[1,z] represent the number of motor imagery categories. It is an indicator function. Indicates when hour, The value is 1, otherwise The value is 0. Let represent the target domain label corresponding to the k-th target synthesized EEG signal. This target domain label is the target domain label corresponding to the target domain EEG signal. log represents the logarithmic function. This represents the EEG signal recognition model, where θ represents the model parameters. This represents the synthesized EEG signal of the k-th target. The corresponding predicted probability, Indicates when hour, The value is 1, otherwise The value is 0. This represents the source domain label corresponding to the k-th target synthesized EEG signal. This source domain label is the source domain label corresponding to the source domain EEG signal.

[0094] Step S207: Based on the model loss value corresponding to the i-th iteration training, adjust the model parameters of the pre-trained EEG signal recognition model to obtain the EEG signal recognition model after the i-th iteration training.

[0095] It should be noted that the method of adjusting model parameters based on model loss value is described in relevant technologies, and will not be repeated in this embodiment.

[0096] This embodiment provides a specific training scheme for the fine-tuning stage of an EEG signal recognition model. By combining the predicted probability output by the EEG signal recognition model with the source domain label and the target domain label, the model loss value corresponding to the current iteration training is determined, thereby adjusting the model parameters to obtain the EEG signal recognition model after iterative training. This allows the EEG signal recognition model to update its own model parameters with each iteration training and gradually converge.

[0097] In this embodiment, step S208 is executed after the i-th iteration of training of the EEG signal recognition model is completed.

[0098] Step S208: Determine whether the EEG signal recognition model after the i-th iteration of training meets the preset convergence condition.

[0099] If it is determined that the EEG signal recognition model after the i-th iteration of training meets the preset convergence condition, then step S209 is executed; if it is determined that the EEG signal recognition model after the i-th iteration of training does not meet the preset convergence condition, then step S210 is executed.

[0100] Step S209: The EEG signal recognition model that meets the preset convergence condition is identified as the trained EEG signal recognition model.

[0101] Among them, the trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

[0102] If the EEG signal recognition model after the i-th iteration of training satisfies the preset convergence condition, it means that the accuracy of the EEG signal model after the i-th iteration of training is high enough. Then, the EEG signal recognition model that satisfies the preset convergence condition can be identified as the EEG signal recognition model that has been trained.

[0103] Step S210: Perform the (i+1)th iteration training on the EEG signal recognition model.

[0104] If it is determined that the EEG signal recognition model after the i-th iteration of training does not meet the preset convergence condition, it means that the accuracy of the EEG signal model after the i-th iteration of training is not high enough, and then the EEG signal recognition model is trained for the (i+1)-th iteration.

[0105] After the EEG signal recognition model is trained, it can be used to identify EEG signals. Optionally, the method provided in this application further includes: acquiring the EEG signal to be identified; inputting the EEG signal to be identified into the trained EEG signal recognition model; using the trained EEG signal recognition model to determine at least two predicted probabilities corresponding to the EEG signal to be identified, each predicted probability representing the probability that the EEG signal to be identified belongs to a corresponding motor imagery category; and determining the motor imagery category corresponding to the highest predicted probability as the target motor imagery category to which the EEG signal to be identified belongs.

[0106] Among them, the EEG signal to be identified is any EEG signal that is to be used for motor image recognition.

[0107] In this embodiment, since the trained EEG signal recognition model has sufficiently high accuracy, the target motor imagery category obtained by recognizing the EEG signal to be recognized through this EEG signal recognition model is relatively accurate.

[0108] This application employs Mix-Tuning (an algorithm for cross-domain knowledge transfer), which generates target synthetic EEG signals through inter-domain Mixup, fine-tunes model parameters, and introduces a source domain data neighborhood risk minimization strategy during the fine-tuning stage. This alleviates the catastrophic forgetting problem in Vanilla Fine-tuning during the fine-tuning stage, improves the model's generalization performance, and enhances accuracy.

[0109] In related technologies, the true data distribution of labeled data in the target domain is usually unknown. Vanilla Fine-tuning searches for the optimal hypothesis based on target domain sample data during the fine-tuning phase using an empirical risk minimization strategy. The empirical risk is shown in Formula 4 below:

[0110] Formula 4:

[0111] Among them, R emp (θ) represents empirical risk, and l represents the cost function. For motion imagery classification tasks, the cross-entropy loss function is often chosen as the cost function.

[0112] However, the labeled data corresponding to the EEG signals required for motor imagery tasks, i.e., the labels corresponding to the EEG signals, are expensive to obtain. Related technologies only use the labeled data of the target domain to fine-tune the model parameters of the EEG signal recognition model using an empirical risk minimization strategy. Since the labeled data of a single subject often cannot provide enough samples for fine-tuning the model parameters of the EEG signal recognition model, and the empirical risk minimization strategy tends to simply memorize the training data, the EEG signal recognition model over-memorizes the labeled data of the target domain, resulting in poor generalization ability.

[0113] This application employs a neighborhood risk minimization strategy, training the EEG signal recognition model based on the neighborhood distribution of source domain sample data and target domain sample data, thus expanding the scale of the training data. Specifically, this application uses an inter-domain mixup data augmentation strategy, convexly combining inter-domain data to construct the neighborhood distribution of source domain sample data and target domain sample data. Subsequently, a new dataset is constructed by randomly sampling from the neighborhood distribution. During the fine-tuning stage, this dataset is used to train the EEG signal recognition model, fine-tuning the model parameters and transforming empirical risk minimization into neighborhood risk minimization, thereby reducing the overfitting risk during model fine-tuning. The neighborhood risk is shown in Formula 5 below:

[0114] Formula 5:

[0115] Among them, R v (θ) represents neighborhood risk. For example, neighborhood risk R... v (θ) can be further divided into target domain neighborhood risk R l (θ) and source-neighborhood risk R s (θ), the target domain neighborhood risk and the source domain neighborhood risk are shown in Formula 6 below:

[0116] Formula Six:

[0117] In the application, the target domain neighborhood risk R l Minimizing (θ) prompts the output data of the EEG signal recognition model to fit the target domain label. Source domain neighborhood risk R s Minimizing (θ) prompts the output data of the EEG signal recognition model to fit the source domain label. Where R... s (θ) can be seen as a regularization term in the fine-tuning stage, which prompts the EEG signal recognition model to favor the effective fitting of source domain labels.

[0118] It should be noted that this application does not impose any restrictions on the model structure of the EEG signal recognition model. The following explanation uses Deep ConvNet as the backbone network of the EEG signal recognition model as an example.

[0119] This application does not limit the network structure of Deep ConvNet. For example, Deep ConvNet consists of four convolutional-pooling blocks and one fully connected layer. The first convolutional-pooling block uses temporal convolution and spatial convolution to extract temporal and spatial features corresponding to the EEG signal, respectively, and then downsamples the extracted features through a max pooling layer. The next three convolutional-pooling blocks consist of temporal convolutional layers and max pooling layers, which can further extract task-related features. The features processed by the four convolutional-pooling blocks are fed into the fully connected layer, where the number of neurons is equal to the number of motor imagery categories. Finally, the predicted probability corresponding to each motor imagery category can be obtained by applying the output of the softmax (normalization exponent) function.

[0120] The training process of the EEG signal recognition model is briefly described below. The data used are source domain sample data and target domain sample data. The model hyperparameter is α, the number of pre-training iterations is T1, the number of fine-tuning iterations is T2, and the optimal parameters obtained after training are... During training, the model parameters θ are randomly initialized. During pre-training, for the current training iteration, source domain sample data is randomly sampled, and the sampled source domain EEG signals are input into the model. The predicted probability is output, and the pre-training loss value is calculated by combining the corresponding source domain labels. The model parameters are then updated via backpropagation. During fine-tuning, for the current training iteration, source domain sample data and target domain sample data are randomly sampled, and the sampled source domain sample data and target domain sample data are convexly combined to obtain the target synthetic EEG signal. The target synthetic EEG signal is input into the model, and the predicted probability is output. The model loss value is calculated by combining the corresponding source domain labels and target domain labels. The model parameters are then updated via backpropagation.

[0121] Figure 4 This is a schematic diagram illustrating a fine-tuning stage according to an exemplary embodiment, such as... Figure 4 As shown, in the data mixing stage, inter-domain interpolation coefficients λ are used to interpolate the target domain EEG signals. Heyuan domain EEG signals Convex combination processing is performed to obtain the target synthetic EEG signal. Synthesize brain signals to target The input is fed into the EEG signal recognition model, whose network structure is based on Deep ConvNet. Deep ConvNet consists of four convolutional layers and one fully connected layer. In the label mixing stage, the cross-entropy loss function is used. And inter-domain interpolation coefficients λ, output data based on EEG signal recognition model, source domain labels and target domain tags The model loss value is determined, and the model parameters of each layer of DeepConvNet are fine-tuned based on the model loss value.

[0122] In this application, the inventors conducted experiments using the motor imagery portion of the BMIdataset dataset (an open-source dataset). This dataset samples the subjects' EEG signals at a frequency of 1000Hz using 62 electrodes and contains left- and right-hand bivariate motor imagery task data from 54 subjects (numbered S1-S54). Each subject's dataset is divided into two sessions, each containing 200 class-balanced samples. The inventors used the first session data from the target subject as the training set for the target domain and the second session data as the test set. All data from the other 53 subjects (not the target subject) were combined to form the source domain sample data.

[0123] To verify the performance advantages of the training method for the EEG signal recognition model provided in this application, comparative experiments were conducted with deep learning models and traditional EEG models. For the deep learning model, due to its powerful feature extraction capabilities, the data underwent the simplest preprocessing. Specifically, EEG signals sampled from all electrodes in the dataset were retained, and the EEG signals collected during the 4-second motor imagery task performed by the subjects were used as input data, with the frequency downsampled to 250Hz. For the traditional EEG model, due to its sensitivity to noise, 20 electrodes selected by the dataset publisher and related to the motor imagery task were used as input signals, and the data sampling frequency was downsampled to 250Hz. Furthermore, to improve the signal-to-noise ratio, a bandpass filter of 8-30Hz was used to filter the data.

[0124] The Mix-Tuning method used in this application is implemented using PyTorch (an open-source Python machine learning library). All experiments were run on the same device, which has an Intel(R) Xeon(R) Silver 4214 CPU and an Nvidia Tesla V100 GPU. The operating system was CentOS Linux release 7.9.2009, kernel version 3.10.0-1160.62.1.el7.x86_64. During the experiments, training was performed using the mini-batch gradient descent algorithm, with the batch size set to 50 except for the experiment involving the number of samples. The optimizer used was Adam (a deep learning optimization algorithm), with corresponding parameters set to 0.9 and 0.99. In the model using the Subject-independent training method, the learning rate and the number of iterations were set to 0.01 and 200, respectively. In the model using the Subject-adaptive training method, the learning rate and number of iterations were set to 0.01 and 200, respectively. For the Mix-Tuning algorithm used in this application, the learning rate and number of iterations were set to 0.01 and 200 during the pre-training phase, and to 0.003 and 300 during the fine-tuning phase. The evaluation metrics were accuracy and F-measure, where the F-measure is the harmonic mean of precision and recall.

[0125] To verify the effectiveness of the Mix-Tuning algorithm, the inventors conducted multiple sets of experiments, comparing it with other algorithms that employed different training methods. These algorithms included several models using Subject-specific, Subject-independent, and Subject-adaptive training methods.

[0126] For subject-specific training, which is used when consistency in the distribution of training and testing data is required, this approach uses only labeled data from the target subjects, avoiding discrepancies in data distribution among different subjects. Specifically, Common Spatial Pattern (CSP) extracts the optimal spatial projection of multi-channel EEG signals; Filter Bank Common Spatial Pattern (FBCSP) divides the EEG data into multiple frequency bands, using CSP to extract features for each band. CSP and Filter Bank Common Spatial Pattern complete the EEG signal data feature extraction task, and Linear Discriminant Analysis (LDA) classifies the extracted features.

[0127] To increase the amount of training data, Subject-independent training merges data from multiple subjects into a single training set, ignoring differences in data distribution among different subjects. This training method is used as a comparison algorithm in Mix-Tuning, and several models employing Subject-independent training include EEGNet, Shallow ConvNet, Deep ConvNet, and TIDNet. Although EEGNet and Shallow ConvNet were originally designed for subject-specific training, they perform better on the BMI dataset using Subject-independent training. These models use default parameter settings except for the dropout rate. The dropout rate for the Dropout layers in EEGNet, Shallow ConvNet, and Deep ConvNet is set to 0.5. TIDNet includes 5 Dropout layers with dropout rates set to {0, 0, 0.4, 0.4, 0}.

[0128] To improve classification accuracy across subjects, Subject-adaptive employs a pre-training and fine-tuning strategy to uncover latent information from non-target subjects. While Subject-adaptive training methods are relatively rare in motor imagery EEG signal classification tasks, to verify the effectiveness of the Mix-Tuning method, the inventors selected several successful pre-training-fine-tuning algorithms in Computer Vision (CV) and Natural Language Processing (NLP) and transferred them to the motor imagery EEG signal classification task for comparison. These models employing Subject-independent training include pre-trained models, Vanilla Fine-tuning, Weight Decay, RecAdam, Child-Tuning, and the Mix-Tuning method used in this application.

[0129] During fine-tuning, Weight Decay incorporates an L2 regularization term into the loss function to penalize larger parameter values. This limits model complexity, prevents overfitting, and improves generalization performance. Specifically, Weight Decay adds the L2 norm of the difference between the model parameters and the pre-trained parameters as a regularization term to the loss function during fine-tuning. In comparative experiments, a grid search method was used to try different hyperparameter values ​​{10, 1, 0.1, 0.01, 0.001} and select the optimal value.

[0130] RecAdam employs a method similar to simulated annealing to reduce the weight of L2 regularization during fine-tuning, thereby improving the model's fine-tuning performance. The RecAdam algorithm's loss function includes an L2 regularization term, the weight of which gradually decreases as fine-tuning progresses. For comparative experiments, a grid search method is used to find the optimal values ​​for the decay rate hyperparameters and the optimal initial values ​​for the algorithm's iterations within the parameter sets {0.05, 0.1, 0.2, 0.5} and {100, 200, 300}.

[0131] To suppress the magnitude of parameter updates, Child-Tuning employs a Dropout-like strategy. Specifically, a probability value *p* is set, and each iteration generates a Bernoulli matrix with the same dimensions as the model parameters, where each element is 1 with probability *p* and 0 with probability (1-p). During parameter updates, only the parameters with a value of 1 in the corresponding Bernoulli matrix are updated. In comparative experiments, different probability values ​​*p*, including {0.1, 0.3, 0.5, 0.7, 0.9}, are tried to find the optimal hyperparameter values.

[0132] In the experiment, Deep ConvNet was used as the backbone network of the Subject-adaptive method, and all parameters except for algorithm-related parameters were kept consistent to ensure a fair comparison of the experimental results of different algorithms. Table 1 shows the accuracy and F-measure of the three training methods (Subject-specific, Subject-independent, and Subject-adaptive) on the BMI dataset for 54 subjects, with the standard deviation in parentheses.

[0133] Table 1

[0134]

[0135] In the motor imagery EEG signal classification task, the response of neurons controlled by the subjects has a significant impact on data quality. Therefore, the accuracy of the same algorithm varies greatly among different subjects, which leads to the large differences in accuracy and F-measure standard deviation of different algorithms on the BMI dataset (54 subjects) motor imagery EEG signal classification task shown in Table 1. The CSP and FBCSP algorithms performed poorly, possibly due to the difference in data distribution between the training and test sets of the target domain data, which come from different sessions. In the subject-independent experiment, Deep ConvNet performed well, achieving an accuracy of 84.01%. EEGNet and Shallow CNN networks are smaller in scale and have weaker fitting ability, resulting in lower prediction accuracy than DeepConvNet. TIDNet has too many parameters, a high risk of overfitting, and a low prediction accuracy. In the subject-adaptive experiment, Pretrain achieved an accuracy of 80.71% on the test data of 54 subjects. Vanilla Fine-tuning, without any regularization strategy, suffered from severe overfitting, with its accuracy dropping to 77.23%. Compared to pretrain, Vanilla Fine-tuning reduced accuracy by 3.48%. Weight Decay, RecAdam, and Child-Tuning mitigated overfitting during fine-tuning by suppressing parameter updates. Their accuracy improved slightly compared to Vanilla Fine-tuning, but remained lower than pretrain. Mix-Tuning achieved 85.50% accuracy by generating synthetic EEG signals through inter-domain mixup for fine-tuning. Compared to pretrain, Mix-Tuning improved accuracy by 4.79%, demonstrating positive knowledge transfer across subjects.

[0136] The following describes the changes in cross-entropy loss function and test set accuracy of each model with the number of iterations in the Subject-adaptive experiment. Vanilla Fine-tuning, lacking regularization, easily falls into overfitting during fine-tuning, causing a rapid decrease in the training set loss function while failing to steadily improve test set accuracy. In contrast, Weight Decay, RecAdam, and Child-Tuning add regularization terms, which alleviates overfitting to some extent, but still suffer from accuracy degradation due to data sparsity. Mix-Tuning, by sampling the neighborhood distribution of the source and target domains, introduces neighborhood risk minimization in the source domain, mitigating the overfitting problem of Vanilla Fine-tuning in sparse sample scenarios and demonstrating better generalization performance on the target domain test data. The accuracy of Mix-Tuning gradually increases and stabilizes during the fine-tuning phase, exhibiting good robustness. In a scenario with 200 labeled samples, Weight Decay, RecAdam, and Child-Tuning actually achieved lower accuracy than Pretrain, while Mix-Tuning effectively addressed the overfitting problem and improved the model's generalization ability.

[0137] To test the performance of Mix-Tuning under data scarcity, the inventors conducted a series of experiments. Due to the high cost of annotated EEG signal data, a subset of the target domain's annotated data was randomly selected, namely {200, 128, 64, 32, 16, 8} balanced samples, to simulate an extremely sparse sample size for the target subjects. To ensure consistent parameter updates during model fine-tuning, the batch size was set to one-quarter of the sample size. That is, when the sample sizes were {200, 128, 64, 32, 16, 8}, the batch sizes were set to {50, 32, 16, 8, 4, 2}, respectively. Table 2 lists the accuracy and standard deviation of different algorithms under different sample sizes.

[0138] Table 2

[0139]

[0140] As shown in Table 2, the accuracy of Vanilla Fine-tuning drops rapidly on the target domain test set as the number of samples decreases, indicating a severe overfitting problem. To mitigate overfitting during fine-tuning, Weight Decay, RecAdam, and Child-Tuning employ strategies to suppress parameter updates during the fine-tuning process. These methods outperform Vanilla Fine-tuning when the sample size is small. However, when the sample size is {128, 64, 32, 16, 8}, the accuracy of Weight Decay, RecAdam, and Child-Tuning after fine-tuning is lower than the accuracy of the pre-trained model, meaning that the fine-tuning process actually leads to a decrease in prediction accuracy. This may be due to the small sample size, making the model prone to overfitting. Therefore, special attention needs to be paid to the risk of model overfitting when the sample size is small. As the sample size decreases, the risk of model overfitting gradually increases, and therefore the accuracy of Mix-Tuning also decreases. Even with only 16 samples, Mix-Tuning still achieved an accuracy of 80.91%, slightly higher than the 80.71% accuracy of the pre-trained model. This demonstrates that Mix-Tuning can handle scenarios with extremely sparse labeled samples in the target domain and achieve positive knowledge transfer across subjects. Therefore, Mix-Tuning is a viable option when the number of samples is very limited.

[0141] The number of layers and parameters in a convolutional neural network (CNN) affects its fitting ability. Increasing the adjustable parameters of a CNN improves its ability to fit downstream tasks, but also increases the risk of overfitting. To test the performance of Mix-Tuning on models with different fitting abilities, the inventors designed an experiment with network adaptive layers. This experiment fixed some parameters of Deep ConvNet by adjusting its fitting ability to downstream tasks. Deep ConvNet consists of four convolutional-pooling blocks and one fully connected layer, and its network parameters are composed of the parameters of the convolutional-pooling blocks and the fully connected layer. In the fine-tuning stage, the parameters of the first m convolutional-pooling blocks of the Deep ConvNet pre-trained model were used to obtain models with different fitting abilities, Frozen-m. The value of m determines the number of adjustable parameters in the model during the fine-tuning stage, affecting the model's fitting ability to downstream tasks. See Table 3, which shows the number of adjustable parameters in Frozen-m in both the input and label spaces.

[0142] Table 3

[0143] Model parameter Number of parameters Frozen-4 <![CDATA[θ c ]]> 2802 Frozen-3 <![CDATA[θ4,θ c ]]> 203202 Frozen-2 <![CDATA[θ3,θ4,θ c ]]> 253402 Frozen-1 <![CDATA[θ2,θ3,θ4,θ c ]]> 266002 Frozen-0 <![CDATA[θ1,θ2,θ3,θ4,θ c ]]> 305077

[0144] In the network adaptive layer experiments, the inventors also observed the performance of different Subject-adaptive algorithms under Frozen-m (see Table 4). Mix-Tuning performed better in Frozen-m, with a more concentrated accuracy distribution and a higher lower bound on accuracy. As m decreases, Frozen-m's ability to fit downstream tasks increases, but the risk of overfitting also increases. However, algorithms such as Vanilla Fine-tuning, Weight Decay, RecAdam, and Child-Tuning, due to the lack of effective regularization strategies, actually saw their accuracy decrease as the adjustable parameters of the model increased. From Frozen-4 to Frozen-0, the average accuracy of Vanilla Fine-tuning, Weight Decay, and Child-Tuning decreased by 2.56%, 3.22%, and 2.15%, respectively. Meanwhile, the average accuracy of Mix-Tuning gradually increased with the improvement of the model's fitting ability. Through experiments with network adaptive layers, Mix-Tuning has been shown to train models with good generalization performance even with a large number of model parameters and a high risk of overfitting.

[0145] Table 4

[0146]

[0147]

[0148] The Mix-Tuning algorithm includes two hyperparameters: the beta distribution parameter α and the learning rate lr during the model phase. The inventors also conducted parameter sensitivity experiments on these two hyperparameters, manually setting different values ​​for α and lr to find the optimal hyperparameters on the BMI dataset, while observing the sensitivity of the Mix-Tuning algorithm to these two hyperparameters. Different values ​​of the beta distribution parameter α affect the probability density function, thus affecting the beta distribution sample values. In this experiment, the learning rate was fixed at 0.003, and the values ​​were set to {0.1, 0.2, 0.5, 1.0, 2.0}. Figure 5 This is a schematic diagram of a line graph according to an exemplary embodiment. Figure 5The variation of Mix-Tuning accuracy with α is shown. The accuracy of Mix-Tuning changes very little with the value of α, indicating that the Mix-Tuning algorithm is not sensitive to the beta distribution parameter. To improve the model's generalization performance, the learning rate is set to 0.01 during the pre-training phase of Mix-Tuning. In the fine-tuning phase, to better fit the target domain data distribution, the learning rate lr should be set to a value less than 0.01. With the distribution parameter fixed at 1, the learning rate lr in the fine-tuning phase is set to {0.01, 0.003, 0.001, 0.0003, 0.0001} respectively. Figure 5 The experiment also demonstrated the fluctuation of Mix-Tuning's accuracy with varying learning rate (lr). The Mix-Tuning algorithm achieved its highest accuracy of 85.50% with a learning rate (lr) of 0.003. When the learning rate was higher than 0.003, the Mix-Tuning algorithm failed to fit the target domain data well; when the learning rate was lower than 0.003, the model parameter update step size was too small, resulting in insufficient training of the model parameters. In other experiments, the learning rate (lr) and the beta distribution parameter (α) were fixed at 0.003 and 1, respectively.

[0149] Figure 6 This is a schematic diagram illustrating the structure of a training device for an electroencephalogram (EEG) signal recognition model according to an exemplary embodiment, such as... Figure 6 As shown, in this embodiment, the training device 300 for the EEG signal recognition model can be installed in an electronic device. The training device 300 for the EEG signal recognition model includes:

[0150] The acquisition module 301 is used to acquire source domain sample data and target domain sample data, and call the pre-trained EEG signal recognition model. The pre-trained EEG signal recognition model is obtained by pre-training based on the source domain sample data. The source domain sample data includes multiple source domain EEG signals and source domain labels corresponding to each source domain EEG signal. The target domain sample data includes multiple target domain EEG signals and target domain labels corresponding to each target domain EEG signal. The source domain labels and target domain labels respectively represent the motor imagery category to which the corresponding EEG signal belongs.

[0151] The sampling module 302 is used to sample from the source domain sample data to obtain the source domain EEG signal and the corresponding source domain label corresponding to the i-th iteration training during the fine-tuning stage of the pre-trained EEG signal recognition model, and to sample from the target source sample data to obtain the target domain EEG signal and the corresponding target domain label corresponding to the i-th iteration training, where i is a positive integer greater than or equal to 1.

[0152] The convex combination processing module 303 is used to perform convex combination processing on the source domain EEG signal and the target domain EEG signal to obtain the target synthetic EEG signal.

[0153] Training module 304 is used to train the pre-trained EEG signal recognition model for the i-th iteration based on the target synthesized EEG signal using source domain labels and target domain labels as supervision data, so as to obtain the EEG signal recognition model after the i-th iteration training.

[0154] The determination module 305 is used to determine the EEG signal recognition model that satisfies the preset convergence condition as the trained EEG signal recognition model if the EEG signal recognition model after the i-th iteration of training satisfies the preset convergence condition. The trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

[0155] Optionally, training module 304 is specifically used for:

[0156] The target synthesized EEG signal is input into the pre-trained EEG signal recognition model. Using the pre-trained EEG signal recognition model, at least two prediction probabilities corresponding to the target synthesized EEG signal are determined. Each prediction probability represents the probability that the target synthesized EEG signal belongs to the corresponding motor imagery category.

[0157] Based on at least two predicted probabilities, source domain labels, and target domain labels corresponding to the target synthesized EEG signal, determine the model loss value corresponding to the i-th iteration of training;

[0158] Based on the model loss value corresponding to the i-th iteration of training, the model parameters of the pre-trained EEG signal recognition model are adjusted to obtain the EEG signal recognition model after the i-th iteration of training.

[0159] Optionally, training module 304 is specifically used for:

[0160] The category of motion imagery corresponding to the highest predicted probability among at least two predicted probabilities is determined as the predicted motion imagery category;

[0161] The cross-entropy function is used to determine the source domain loss value between the predicted motion imagery category and the source domain label, and the cross-entropy function is used to determine the target domain loss value between the predicted motion imagery category and the target domain label.

[0162] The source domain loss value and the target domain loss value are weighted and summed to obtain the model loss value corresponding to the i-th iteration training.

[0163] Optionally, the convex combination processing module 303 is specifically used for:

[0164] Obtain multiple inter-domain interpolation coefficients that follow a beta distribution;

[0165] Using the inter-domain interpolation coefficients as weights, the corresponding source domain EEG signals and the corresponding target domain EEG signals are weighted and summed to obtain the neighborhood distributions of the source domain EEG signals and the target domain EEG signals. The neighborhood distributions include multiple synthetic EEG signals.

[0166] Random sampling is performed from the neighborhood distribution to obtain the target synthetic EEG signal.

[0167] Optionally, the convex combination processing module 303 is specifically used for:

[0168] Using the inter-domain interpolation coefficients as weights, the following formula is used to perform weighted summation on the corresponding source domain EEG signals and the corresponding target domain EEG signals, respectively:

[0169]

[0170]

[0171] To obtain the neighborhood distribution of source domain EEG signals and target domain EEG signals;

[0172] Among them, P v Indicates neighborhood distribution, Indicates synthesized EEG signals, Represents the target domain EEG signal. Represents the source domain EEG signal, i∈[1,n] s ],j∈[1,n l ], n s n represents the number of source domain EEG signals included in the source domain sample data. l This represents the number of target domain EEG signals included in the target domain sample data, where λ is the inter-domain data interpolation coefficient, λ ~ Beta(α,α), α∈(0,+∞]. Let denot represent the values ​​of all λ, and δ represent the Dirac function.

[0173] Optionally, the device 300 further includes:

[0174] The acquisition module 301 is also used to acquire the EEG signal to be identified;

[0175] The determination module 305 is also used to input the EEG signal to be identified into the trained EEG signal recognition model, and use the trained EEG signal recognition model to determine at least two predicted probabilities corresponding to the EEG signal to be identified, each predicted probability representing the probability that the EEG signal to be identified belongs to the corresponding motor imagery category.

[0176] The determination module 305 is also used to determine the motor imagery category corresponding to the maximum predicted probability as the target motor imagery category to which the EEG signal to be identified belongs.

[0177] The training device for the EEG signal recognition model provided in this embodiment can execute the technical solution of the method embodiment shown. Its implementation principle and technical effect are similar to those of the method embodiment shown, and will not be described in detail here.

[0178] This application also provides an electronic device. The electronic device is intended for various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers.

[0179] Figure 7 This is a schematic diagram illustrating the structure of an electronic device according to an exemplary embodiment. For example... Figure 7 As shown, the electronic device 400 includes a processor 401 and a memory 402 communicatively connected to the processor 401.

[0180] The memory 402 stores computer execution instructions; the processor 401 executes the computer execution instructions stored in the memory 402 to implement the training method of the EEG signal recognition model provided in this application.

[0181] In this embodiment, the memory 402 and the processor 401 are connected via a bus. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one thick line is used in the figure, but this does not indicate that there is only one bus or one type of bus.

[0182] The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise as required.

[0183] In an exemplary embodiment, a computer-readable storage medium is also provided, which stores computer-executable instructions that, when executed by a processor, are used to implement the training method for the EEG signal recognition model provided in this application.

[0184] In an exemplary embodiment, a computer program product is also provided, including computer execution instructions, which, when executed by a processor, are used to implement the training method for the EEG signal recognition model provided in this application.

[0185] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0186] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A training method for an electroencephalogram (EEG) signal recognition model, characterized in that, include: The process involves acquiring source domain sample data and target domain sample data, and then calling a pre-trained EEG signal recognition model. The pre-trained EEG signal recognition model is obtained based on the source domain sample data, which includes multiple source domain EEG signals and corresponding source domain labels. The target domain sample data includes multiple target domain EEG signals and corresponding target domain labels. The source domain labels and target domain labels represent the motor imagery category to which the corresponding EEG signal belongs. During the i-th iteration of fine-tuning the pre-trained EEG signal recognition model, samples are taken from the source domain sample data to obtain the source domain EEG signal and corresponding source domain label corresponding to the i-th iteration of training, and samples are taken from the target source sample data to obtain the target domain EEG signal and corresponding target domain label corresponding to the i-th iteration of training, where i is a positive integer greater than or equal to 1. Multiple inter-domain interpolation coefficients following a beta distribution are obtained; using each inter-domain interpolation coefficient as a weight, the corresponding source domain EEG signal and the corresponding target domain EEG signal are weighted and summed to obtain the neighborhood distributions corresponding to the source domain EEG signal and the target domain EEG signal, wherein the neighborhood distributions include multiple synthetic EEG signals; random sampling is performed from the neighborhood distributions to obtain the target synthetic EEG signal; Using the source domain label and the target domain label as supervision data, the pre-trained EEG signal recognition model is trained for the i-th iteration based on the target synthesized EEG signal to obtain the EEG signal recognition model after the i-th iteration training. If it is determined that the EEG signal recognition model after the i-th iteration of training meets the preset convergence condition, then the EEG signal recognition model that meets the preset convergence condition is determined as the trained EEG signal recognition model. The trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

2. The method according to claim 1, characterized in that, The step of using the source domain label and the target domain label as supervision data, and performing the i-th iteration training on the pre-trained EEG signal recognition model based on the target synthesized EEG signal to obtain the EEG signal recognition model after the i-th iteration training, includes: The target synthetic EEG signal is input into the pre-trained EEG signal recognition model. Using the pre-trained EEG signal recognition model, at least two prediction probabilities corresponding to the target synthetic EEG signal are determined. Each prediction probability represents the probability that the target synthetic EEG signal belongs to the corresponding motor imagery category. Based on the at least two predicted probabilities corresponding to the target synthesized EEG signal, the source domain label, and the target domain label, the model loss value corresponding to the i-th iteration training is determined; Based on the model loss value corresponding to the i-th iteration of training, the model parameters of the pre-trained EEG signal recognition model are adjusted to obtain the EEG signal recognition model after the i-th iteration of training.

3. The method according to claim 2, characterized in that, The step of determining the model loss value corresponding to the i-th iteration of training based on the at least two predicted probabilities corresponding to the target synthesized EEG signal, the source domain label, and the target domain label includes: The category of motion imagery corresponding to the highest prediction probability among the at least two prediction probabilities is determined as the predicted motion imagery category; The cross-entropy function is used to determine the source domain loss value between the predicted motion imagery category and the source domain label, and the cross-entropy function is used to determine the target domain loss value between the predicted motion imagery category and the target domain label. The source domain loss value and the target domain loss value are weighted and summed to obtain the model loss value corresponding to the i-th iteration of training.

4. The method according to claim 1, characterized in that, The step of using the inter-domain interpolation coefficients as weights to perform weighted summation on the corresponding source domain EEG signals and the corresponding target domain EEG signals to obtain the neighborhood distributions of the source domain EEG signals and the target domain EEG signals includes: Using the inter-domain interpolation coefficients as weights, the following formulas are used to perform weighted summation on the corresponding source domain EEG signals and the corresponding target domain EEG signals, respectively: To obtain the neighborhood distribution of the source domain EEG signal and the target domain EEG signal; in, Indicates the neighborhood distribution, This indicates the synthesized EEG signal. This represents the target domain EEG signal. This represents the source domain EEG signal. , , This indicates the number of source domain EEG signals included in the source domain sample data. This indicates the number of target domain EEG signals included in the target domain sample data. These are the inter-domain data interpolation coefficients. , , Representing the whole The value of , This represents the Dirac function.

5. The method according to claim 1, characterized in that, The method further includes: Acquire the EEG signal to be identified; The EEG signal to be identified is input into the trained EEG signal recognition model. Using the trained EEG signal recognition model, at least two predicted probabilities corresponding to the EEG signal to be identified are determined. Each predicted probability represents the probability that the EEG signal to be identified belongs to the corresponding motor imagery category. The category of motor imagery corresponding to the highest predicted probability is determined as the target motor imagery category to which the EEG signal to be identified belongs.

6. A training device for an electroencephalogram (EEG) signal recognition model, characterized in that, include: The acquisition module is used to acquire source domain sample data and target domain sample data, and call a pre-trained EEG signal recognition model; wherein, the pre-trained EEG signal recognition model is obtained by pre-training based on the source domain sample data, the source domain sample data includes multiple source domain EEG signals and source domain labels corresponding to each source domain EEG signal, and the target domain sample data includes multiple target domain EEG signals and target domain labels corresponding to each target domain EEG signal, wherein the source domain label and the target domain label respectively represent the motor imagery category to which the corresponding EEG signal belongs; The sampling module is used to sample from the source domain sample data during the i-th iteration of fine-tuning the pre-trained EEG signal recognition model to obtain the source domain EEG signal and corresponding source domain label corresponding to the i-th iteration of training, and to sample from the target source sample data to obtain the target domain EEG signal and corresponding target domain label corresponding to the i-th iteration of training, where i is a positive integer greater than or equal to 1. A convex combination processing module is used to obtain multiple inter-domain interpolation coefficients that follow a beta distribution; using each of the inter-domain interpolation coefficients as weights, weighted summation processing is performed on the corresponding source domain EEG signal and the corresponding target domain EEG signal to obtain the neighborhood distribution corresponding to the source domain EEG signal and the target domain EEG signal, the neighborhood distribution including multiple synthetic EEG signals; random sampling is performed from the neighborhood distribution to obtain the target synthetic EEG signal; The training module is used to perform the i-th iteration training on the pre-trained EEG signal recognition model based on the target synthesized EEG signal using the source domain label and the target domain label as supervision data, so as to obtain the EEG signal recognition model after the i-th iteration training. The determination module is used to determine the EEG signal recognition model that satisfies the preset convergence condition as the trained EEG signal recognition model if it is determined that the EEG signal recognition model after the i-th iteration of training satisfies the preset convergence condition. The trained EEG signal recognition model is used to determine the motor imagery category to which the input EEG signal belongs.

7. An electronic device, characterized in that, include: A processor and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the training method for the EEG signal recognition model as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the training method for the EEG signal recognition model as described in any one of claims 1 to 5.

9. A computer program product comprising computer-executable instructions, characterized in that, When the computer execution instructions are executed by the processor, the training method for the EEG signal recognition model as described in any one of claims 1 to 5 is implemented.

Images