Electroencephalogram signal classification method, device and equipment and storage medium

Abstract

The application is an electroencephalogram signal classification method, device, equipment and storage medium, and relates to the technical field of signal processing. The method comprises the following steps: acquiring a first electroencephalogram signal; processing the first electroencephalogram signal through at least two electroencephalogram signal classification models respectively to obtain motor imagination probability distributions respectively output by the at least two electroencephalogram signal classification models; and determining a motor imagination type corresponding to the first electroencephalogram signal based on the motor imagination probability distributions respectively output by the at least two electroencephalogram signal classification models. The above scheme improves the training effect of the model under the condition of a small amount of training samples by dividing the samples into multiple subsets and then performing data augmentation, and improves the accuracy of electroencephalogram signal classification by simultaneously considering the outputs of multiple electroencephalogram signal classification models when classifying the electroencephalogram signal.

Description

Technical Field

[0001] This application relates to the field of signal processing, and in particular to a method, apparatus, device and storage medium for classifying electroencephalogram (EEG) signals. Background Technology

[0002] Electroencephalogram (EEG) is a method of recording brain activity using electrophysiological indicators. It is formed by summing the postsynaptic potentials that occur synchronously among a large number of neurons during brain activity. It records the changes in electrical waves during brain activity and is a comprehensive reflection of the electrophysiological activity of brain nerve cells on the surface of the cerebral cortex or scalp.

[0003] Among related technologies, MI-BCI (Motor Imagery-Brain Computer Interface) systems have broad application prospects in many fields. They enable control of external devices through brainwave signals generated by imagining limb movements, without any actual physical movement. It can assist patients with limb disabilities such as stroke and hemiplegia in rehabilitation training or wheelchair control, and can also be used for education and entertainment for ordinary users, such as brain-controlled VR (Virtual Reality) games. MI (Motor Imagery) signal classification and recognition is a crucial component of the MI-BCI system; its decoding accuracy directly affects the system's performance and user experience.

[0004] In the above technical solutions, the number of EEG signal samples is small, resulting in poor accuracy of the trained EEG signal classification model and low accuracy in classifying EEG signals. Summary of the Invention

[0005] This application provides a method, apparatus, device, and storage medium for classifying electroencephalogram (EEG) signals, which can improve the accuracy of predicting the type of motor imagery corresponding to EEG signals. The technical solution is as follows:

[0006] On the one hand, a method for classifying electroencephalogram (EEG) signals is provided, the method comprising:

[0007] Acquire the first EEG signal;

[0008] The first EEG signal is processed by at least two EEG signal classification models to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively. The EEG signal classification models are machine learning models trained on an augmented dataset obtained by augmenting a subset of training samples. The training subset includes sample EEG signals in a first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models. The first training sample set includes at least two of the sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification models. The verification EEG signals corresponding to the at least two EEG signal classification models in the first training sample set are different.

[0009] Based on the probability distribution of motor imagery output by at least two of the aforementioned EEG signal classification models, the type of motor imagery corresponding to the first EEG signal is determined.

[0010] On another front, a method for classifying electroencephalogram (EEG) signals is provided, the method comprising:

[0011] Obtain a first training sample set; the first training sample set includes at least two sample EEG signals and at least two motor imagery types corresponding to the sample EEG signals;

[0012] Based on the first training sample set and at least two EEG signal classification models, training sample subsets corresponding to at least two EEG signal classification models are obtained respectively; the training sample subsets include the first training sample set, and the training sample subsets include sample EEG signals in the first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models; the sample EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set; the first training sample set includes at least two sample EEG signals and motor imagery types corresponding to the at least two sample EEG signals; the verification EEG signals are sample EEG signals used to verify the EEG signal classification models; the verification EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set.

[0013] Based on at least two augmented datasets obtained by augmenting at least two subsets of the training samples, at least two EEG signal classification models corresponding to at least two subsets of the training samples are trained to obtain at least two trained EEG signal classification models.

[0014] The at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively, and to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distributions output by the at least two EEG signal classification models respectively.

[0015] In another aspect, an electroencephalogram (EEG) signal classification device is provided, the device comprising:

[0016] The EEG signal acquisition module is used to acquire the first EEG signal;

[0017] A probability distribution acquisition module is used to process the first EEG signal using at least two EEG signal classification models to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively. The EEG signal classification models are machine learning models trained on an augmented dataset obtained by augmenting a subset of training samples. The training sample subset includes sample EEG signals in a first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models. The first training sample set includes at least two of the sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification models. The verification EEG signals corresponding to the at least two EEG signal classification models in the first training sample set are different.

[0018] The motor imagery type acquisition module is used to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distributions output by at least two of the EEG signal classification models.

[0019] In one possible implementation, the motion imagination type acquisition module includes:

[0020] The EEG probability distribution acquisition submodule is used to acquire the motor imagery probability distribution corresponding to the first EEG signal based on the motor imagery probability distributions output by at least two EEG signal classification models respectively; the probability distribution corresponding to the first EEG signal contains probability values ​​corresponding to each type of motor imagery.

[0021] The motor imagery type acquisition submodule is used to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distribution corresponding to the first EEG signal.

[0022] In one possible implementation, the EEG probability distribution acquisition module is further used for:

[0023] The probability distributions of motor imagery output by at least two of the EEG signal classification models are merged based on the type of motor imagery to obtain the probability distribution of motor imagery corresponding to the first EEG signal.

[0024] In one possible implementation, the probability distribution of motor imagery output by the EEG signal classification model includes probability values ​​corresponding to each type of motor imagery.

[0025] The EEG probability distribution acquisition module is also used for,

[0026] The probability distributions of motor imagery output by at least two of the EEG signal classification models are weighted and summed based on the type of motor imagery to obtain the probability distribution of motor imagery corresponding to the first EEG signal.

[0027] In one possible implementation, the motion imagination type acquisition module is further used to,

[0028] The type of motor imagery corresponding to the highest probability value in the motor imagery probability distributions output by at least two of the aforementioned EEG signal classification models is determined as the type of motor imagery corresponding to the first EEG signal.

[0029] In one possible implementation, in response to at least two of the EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model including a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer;

[0030] The probability distribution acquisition module includes:

[0031] The first weighting submodule is used to process the first EEG signal through the first channel attention weighting module to obtain a first weighted feature map.

[0032] The first time extraction submodule is used to process the first weighted feature map through the first time convolutional layer to obtain the first time feature map; the first time convolutional layer is used to extract the temporal features of the EEG signal.

[0033] The first spatial extraction submodule is used to process the first temporal feature map through the first spatial convolutional layer to obtain the first spatial feature map; the first spatial convolutional layer is used to extract the spatial features of different regions of the head of the object corresponding to the EEG signal.

[0034] The first activation submodule is used to process data through the first activation layer based on the first spatial feature map to obtain the first activation feature map.

[0035] The first probability distribution acquisition submodule is used to obtain the probability distribution of motor imagery corresponding to the first EEG signal output by the first EEG signal classification model by processing the data through the first fully connected layer based on the first activation feature map.

[0036] In one possible implementation, the device further includes:

[0037] The first sample set acquisition module is used to acquire the first training sample set;

[0038] The first sample subset acquisition module is used to acquire a first training sample subset based on the sample EEG signals in the first training sample set, excluding the verification EEG signals corresponding to the first EEG signal classification model; the first training sample subset includes the first sample EEG signals.

[0039] The data augmentation module is used to augment the data based on the first training sample subset to obtain a first augmented dataset; the first augmented dataset includes the first sample EEG signal, the first augmented signal corresponding to the first sample EEG signal, and the motor imagery type corresponding to the first sample EEG signal.

[0040] The first training module is used to train the first EEG signal classification model based on the first augmented dataset.

[0041] In one possible implementation, the data augmentation module includes:

[0042] An augmentation factor acquisition submodule is used to acquire the augmentation factor corresponding to the first training sample subset, wherein the augmentation factor is used to indicate the scaling ratio of the samples in the first training sample subset.

[0043] The data augmentation submodule is used to scale the EEG signal of the first sample based on the augmentation factor corresponding to the first training sample subset to obtain the first augmented signal corresponding to the EEG signal of the first sample.

[0044] In one possible implementation, the device further includes:

[0045] The verification signal acquisition module is used to determine the verification EEG signal corresponding to the first EEG signal classification model from at least two sample EEG signals in the first training sample set, based on the first EEG signal classification model.

[0046] In another aspect, an electroencephalogram (EEG) signal classification device is provided, the device comprising:

[0047] The training subset acquisition module is used to acquire a first training sample set; the first training sample set includes at least two sample EEG signals and at least two motor imagery types corresponding to the sample EEG signals.

[0048] A training sample subset acquisition module is used to acquire training sample subsets corresponding to at least two EEG signal classification models, based on the first training sample set and at least two EEG signal classification models. The training sample subsets include the sample EEG signals in the first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models. The sample EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set. The first training sample set includes at least two sample EEG signals and motor imagery types corresponding to the at least two sample EEG signals. The verification EEG signals are sample EEG signals used to verify the EEG signal classification models. The verification EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set.

[0049] The model training module is used to train at least two EEG signal classification models corresponding to at least two training sample subsets based on at least two augmented datasets obtained after data augmentation of at least two training sample subsets, so as to obtain at least two trained EEG signal classification models.

[0050] The at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively, and to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distributions output by the at least two EEG signal classification models respectively.

[0051] In one possible implementation, the training sample subset acquisition module includes:

[0052] The verification signal acquisition submodule is used to determine the verification EEG signal corresponding to the at least two EEG signal classification models respectively in at least two sample EEG signals in the first training sample set, based on at least two EEG signal classification models.

[0053] The training sample subset acquisition submodule is used to acquire training sample subsets corresponding to at least two of the EEG signal classification models based on the verification EEG signals corresponding to at least two of the EEG signal classification models, and the first training sample set.

[0054] In another aspect, a computer-readable storage medium is provided, wherein at least one instruction, at least one program, code set, or instruction set is stored therein, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the above-described EEG signal classification method.

[0055] In another aspect, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the aforementioned electroencephalogram (EEG) signal classification method.

[0056] The beneficial effects of the technical solutions provided in this application include at least the following:

[0057] This approach trains multiple EEG signal classification models by augmenting different subsets of the same training sample set. These augmented subsets are then used to process EEG signals, yielding probability distributions from the outputs of each model. The corresponding motor imagery type is then determined based on these probability distributions. This method, by taking different subsets from a single training sample set and augmenting them to train at least two EEG signal classification models, and then using these models to classify EEG signals to determine their corresponding motor imagery type, improves training efficiency even with a limited number of training samples. Furthermore, by considering the outputs of multiple classification models simultaneously during EEG signal classification, the accuracy of the classification is enhanced. Attached Figure Description

[0058] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0059] Figure 1 A schematic diagram of a computer system provided in an exemplary embodiment of this application is shown;

[0060] Figure 2 This is a flowchart illustrating an electroencephalogram (EEG) signal classification method according to an exemplary embodiment;

[0061] Figure 3 This is a flowchart illustrating an electroencephalogram (EEG) signal classification method according to an exemplary embodiment;

[0062] Figure 4 This is a flowchart illustrating a method for classifying electroencephalogram (EEG) signals according to an exemplary embodiment;

[0063] Figure 5 It shows Figure 4 The illustrated embodiment is a schematic diagram of a training sample subset acquisition method.

[0064] Figure 6 It shows Figure 4 The illustrated embodiment is a schematic diagram of the EEG signal classification model.

[0065] Figure 7 It shows Figure 4 The illustrated embodiment is a schematic diagram of the channel attention mechanism.

[0066] Figure 8 It shows Figure 4 The illustrated embodiment is a schematic diagram of the combined probability distribution of motion imagination.

[0067] Figure 9 This is a flowchart illustrating an example of electroencephalogram (EEG) signal classification.

[0068] Figure 10 This is a structural block diagram of an electroencephalogram (EEG) signal classification device according to an exemplary embodiment;

[0069] Figure 11 This is a structural block diagram of an electroencephalogram (EEG) signal classification device according to an exemplary embodiment;

[0070] Figure 12 This is a schematic diagram of the structure of a computer device according to an exemplary embodiment. Detailed Implementation

[0071] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0072] First, the terms used in the embodiments of this application will be introduced.

[0073] 1) Artificial Intelligence (AI)

[0074] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0075] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0076] 2) Machine Learning (ML)

[0077] Machine learning is a multidisciplinary field involving probability theory, statistics, approximation theory, convex analysis, and algorithm complexity theory, among others. It specifically studies how computers can simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to endow computers with intelligence; its applications span all areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and learn-by-doing.

[0078] 3) Channel

[0079] In convolutional neural networks (CNNs), channels are used to indicate feature maps. The strength of a point within a channel represents the numerical value of the feature map at that point. Different channels indicate feature maps of different dimensions. A feature map with multiple channels signifies that it possesses image features across multiple dimensions. CNNs primarily involve two operations: convolution and pooling. Pooling layers do not affect the interaction between channels; they operate only within individual channels. Convolutional layers, on the other hand, allow interaction between channels before the next layer generates new channels.

[0080] The EEG signal classification method provided in this application can be applied to computer devices with strong data processing capabilities. In one possible implementation, the EEG signal classification method provided in this application can be applied to a personal computer, workstation, or server; that is, the EEG signal classification model can be trained using a personal computer, workstation, or server. In another possible implementation, the EEG signal classification model trained using the EEG signal classification method provided in this application can be used to classify EEG signals, that is, to process the EEG signals generated by the head during human motor imagery to obtain the corresponding motor imagery type.

[0081] Please refer to Figure 1 The diagram illustrates a computer system provided in an exemplary embodiment of this application. The computer system 200 includes a terminal 110 and a server 120, wherein the terminal 110 and the server 120 communicate via a communication network. Optionally, the communication network can be a wired network or a wireless network, and the communication network can be at least one of a local area network (LAN), a metropolitan area network (MAN), and a wide area network (WAN).

[0082] The terminal 110 is equipped with an application with brainwave signal processing capabilities. This application may be a virtual reality application, a game application, or an artificial intelligence (AI) application with brainwave signal processing capabilities. This application embodiment does not limit the scope of the application.

[0083] Optionally, the computer device 110 may be a terminal device with a brain-computer interface, which can acquire the electroencephalogram (EEG) signals of the target object's head through electrodes; or the computer device may have a data transmission interface for receiving EEG signals acquired by a data acquisition device with a brain-computer interface.

[0084] Optionally, the computer device 110 can be a mobile terminal such as a smartphone, tablet, or laptop computer, or a terminal such as a desktop computer or projector computer, or a smart terminal with data processing components. This application embodiment does not limit this.

[0085] Server 120 can be implemented as a single server or as a server cluster consisting of a group of servers. It can be a physical server or a cloud server. In one possible implementation, server 120 is the backend server for the application in computer device 110.

[0086] In one possible implementation of this embodiment, server 120 trains an EEG signal classification model using a pre-set training sample set (i.e., sample EEG signals). The training sample set may contain sample EEG signals corresponding to various types of motor imagery. Once server 120 has completed the training process, it sends the trained EEG signal classification model to terminal 110 via a wired or wireless connection. Terminal 110 receives the trained EEG signal classification model and inputs the corresponding data information into an application with EEG signal processing capabilities. This allows the user to process EEG signals using the application based on the trained EEG signal classification model, thereby implementing all or part of the steps of the EEG signal classification method.

[0087] Figure 2 This is a flowchart illustrating a brainwave signal classification method according to an exemplary embodiment. The method can be executed by a computer device, wherein the computer device can be the one described above. Figure 1 Terminal 120 in the illustrated embodiment. For example... Figure 2 As shown, the process of this EEG signal classification method may include the following steps:

[0088] Step 201: Obtain the first EEG signal.

[0089] In one possible implementation, the first EEG signal is the brainwave signal of the target object acquired by a device with a brain-computer interface. The brain-computer interface has at least two electrodes, which are located in different areas of the target object's head during the signal acquisition process through the brain-computer interface, so as to acquire the brainwave signals generated in different areas of the target object.

[0090] Step 202: Process the first EEG signal using at least two EEG signal classification models to obtain the probability distribution of motor imagery output by at least two EEG signal classification models.

[0091] The EEG signal classification model is a machine learning model obtained by training an augmented dataset obtained by augmenting a subset of training samples. The subset of training samples includes sample EEG signals in a first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification model. The first training sample set includes at least two of the sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification model. The verification EEG signals corresponding to the at least two EEG signal classification models in the first training sample set are different.

[0092] Step 203: Based on the probability distribution of motor imagery output by at least two EEG signal classification models, determine the type of motor imagery corresponding to the first EEG signal.

[0093] In summary, the scheme described in this application trains multiple EEG signal classification models by performing data augmentation on different training subsets of the same training sample set. These models are then used to process EEG signals, yielding probability distributions output by each model. The corresponding motor imagery type is then determined based on these probability distributions. This scheme, by taking different subsets of a training sample set and performing data augmentation to train at least two EEG signal classification models, and then using these models to classify EEG signals to determine their corresponding motor imagery type, improves the training effect of the model when the training sample size is small. Furthermore, by considering the outputs of multiple EEG signal classification models simultaneously during EEG signal classification, the accuracy of EEG signal classification is enhanced.

[0094] Figure 3 This is a flowchart illustrating a brainwave signal classification method according to an exemplary embodiment. The method can be executed by a computer device, wherein the computer device can be the one described above. Figure 1 Server 120 in the illustrated embodiment. For example... Figure 3 As shown, the process of this EEG signal classification method may include the following steps:

[0095] Step 301: Obtain a first training sample set; the first training sample set includes at least two EEG signals of the sample and at least two motor imagery types corresponding to the EEG signals of the sample.

[0096] Step 302: Based on the first training sample set and at least two EEG signal classification models, obtain at least two training sample subsets corresponding to the EEG signal classification models respectively.

[0097] The training sample subset includes the first training sample set, which contains sample EEG signals excluding the verification EEG signals corresponding to the EEG signal classification model. At least two EEG signal classification models correspond to different sample EEG signals in the first training sample set. The first training sample set includes at least two sample EEG signals and at least two corresponding motor imagery types. The verification EEG signal is a sample EEG signal used to verify the EEG signal classification model. At least two EEG signal classification models correspond to different verification EEG signals in the first training sample set.

[0098] Step 303: Based on at least two subsets of the training samples, train at least two EEG signal classification models corresponding to each of the at least two subsets of the training samples to obtain at least two trained EEG signal classification models.

[0099] The at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively, and to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distributions output by the at least two EEG signal classification models respectively.

[0100] In summary, the scheme described in this application trains multiple EEG signal classification models by performing data augmentation on different training subsets of the same training sample set. These models are then used to process EEG signals, yielding probability distributions output by each model. The corresponding motor imagery type is then determined based on these probability distributions. This scheme, by taking different subsets of a training sample set and performing data augmentation to train at least two EEG signal classification models, and then using these models to classify EEG signals to determine their corresponding motor imagery type, improves the training effect of the model with a limited number of training samples by dividing the sample set into multiple subsets and performing data augmentation. Furthermore, by considering the outputs of multiple EEG signal classification models simultaneously during EEG signal classification, the accuracy of EEG signal classification is improved.

[0101] Figure 4 This is a flowchart illustrating a method for classifying electroencephalogram (EEG) signals according to an exemplary embodiment. The method can be executed jointly by a model training device and a signal processing device, wherein the model training device can be the one described above. Figure 1 The server 120 in the illustrated embodiment may be the signal processing device described above. Figure 1 Terminal 120 in the illustrated embodiment. (As shown) Figure 4 As shown, the process of this EEG signal classification method may include the following steps:

[0102] Step 401: Obtain the first training sample set.

[0103] The first training sample set includes at least two EEG signals of the sample and at least two motor imagery types corresponding to the EEG signals of the sample.

[0104] In one possible implementation, the sample EEG signal includes signals from at least two sample electrodes.

[0105] At least two of the sample electrode signals can be brainwave signals generated by the head of the target object during motor imagery, acquired through electrodes of the brain-computer interface via a terminal device with a brain-computer interface. The number of the first sample electrode signals is the same as the number of electrodes corresponding to the brain-computer interface, meaning the brain-computer interface can acquire brainwave signals generated by different spatial regions of the head of the same target object during motor imagery using different electrodes.

[0106] In one possible implementation, the brain-computer interface acquires electroencephalogram (EEG) signals generated in different regions of the head of the target sample through electrodes connected to the target sample, and the EEG signals corresponding to each electrode are transmitted to the terminal device corresponding to the brain-computer interface through a transmission line.

[0107] In one possible implementation, based on the electrodes of the brain-computer interface, the original sample EEG signal generated by the head of the target object during motor imagery is acquired; based on the original sample EEG signal, it is filtered by a bandpass filter to obtain the first sample EEG signal.

[0108] Since the raw EEG signals obtained through the electrodes of the brain-computer interface contain a lot of noise interference, it is necessary to first filter the raw EEG signals through a bandpass filter to reduce the influence of irrelevant noise on the EEG signals.

[0109] Each original sample EEG signal was subjected to a 3-38Hz bandpass filter to remove the effects of irrelevant physiological noise such as eye movement and power frequency interference (i.e., interference caused by the power system, usually 50Hz) on the EEG signal.

[0110] Step 402: Based on the first training sample set and at least two EEG signal classification models, obtain at least two training sample subsets corresponding to the EEG signal classification models respectively.

[0111] The training sample subset includes a first training sample set containing sample EEG signals excluding the verification EEG signals corresponding to the EEG signal classification model; at least two EEG signal classification models have different sample EEG signals in the first training sample set; the first training sample set includes at least two sample EEG signals and at least two motor imagery types corresponding to the sample EEG signals; the verification EEG signal is a sample EEG signal used to verify the EEG signal classification model from the at least two sample EEG signals; at least two EEG signal classification models have different verification EEG signals in the first training sample set.

[0112] In one possible implementation, in response to at least two training sample subsets containing a first training sample subset, the first training sample subset is obtained; based on the first training sample subset, a sample EEG signal other than the verification EEG signal corresponding to the first EEG signal classification model is obtained; wherein the first training sample subset is different from the sample EEG signals corresponding to each of the EEG signal classification models.

[0113] Before training each EEG signal classification model, sample EEG signals corresponding to each EEG signal classification model can be selected from the first training sample set and used as the verification EEG signals corresponding to each EEG signal classification model, so as to verify the trained EEG signal classification model and determine whether the EEG signal classification model has been trained.

[0114] Furthermore, for any one of the EEG signal classification models, the sample EEG signals in the first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification model, can be used as the training samples corresponding to the EEG signal classification model to train the EEG signal classification model.

[0115] In one possible implementation, the verification EEG signal corresponding to the EEG signal classification model can be one of the sample EEG signals; or, the verification EEG signal corresponding to the EEG signal classification model can be multiple of the sample EEG signals. In this case, the EEG signal classification model can obtain the sample EEG signals other than the multiple verification EEG signals in the first training sample set as the training sample subset of the EEG signal classification model, and train based on the training sample subset. The EEG signal classification model after training for a predetermined number of rounds can be verified by the multiple verification EEG signals.

[0116] In one possible implementation, based on at least two EEG signal classification models, a verification EEG signal corresponding to each of the at least two sample EEG signals in the first training sample set is determined; based on the verification EEG signal corresponding to each of the at least two EEG signal classification models and the first training sample set, a training sample subset corresponding to each of the at least two EEG signal classification models is obtained.

[0117] In one possible implementation, in response to the inclusion of a first EEG signal classification model in at least two EEG signal classification models, a verification EEG signal corresponding to the first EEG signal classification model is determined from at least two sample EEG signals in the first training sample set based on the first EEG signal classification model.

[0118] The first training sample subset is one of the subsets of the first training sample set, and the first training sample subset is used to train the first EEG signal classification model. At this time, according to the first EEG signal classification model, the sample EEG signals in the first training sample set that correspond to the first EEG signal classification model are determined, and the sample EEG signals in the first training sample set that do not correspond to the sample EEG signals corresponding to the first EEG signal classification model are obtained as the first training sample subset. Since the first training sample set is different from the verification EEG signals corresponding to each EEG signal classification model, the sample EEG signals that do not correspond to the sample EEG signals corresponding to each EEG signal classification model can be obtained as the training sample subsets corresponding to each EEG signal classification model. At this time, the training sample subsets corresponding to each EEG signal classification model are different.

[0119] In the first training sample set, the verification sample EEG signals corresponding to each EEG signal classification model can be determined according to each EEG signal classification model. The other sample EEG signals in the first training sample set, excluding the verification sample EEG signals corresponding to the EEG signal classification model, are determined as the training sample subset corresponding to the EEG signal classification model. This allows the EEG signal classification model to be trained using the training sample subset and then verified using the verification sample EEG signals.

[0120] Figure 5 This illustration shows a schematic diagram of a training sample subset acquisition method according to an embodiment of this application. Figure 5As shown, in the first training sample set 500, there are first sample EEG signal 501, second sample EEG signal 502, ..., N-1 sample EEG signal 503, and Nth sample EEG signal 504. For the first EEG signal classification model, the Nth sample EEG signal 504 is taken as the verification EEG signal corresponding to the first EEG signal classification model. Therefore, the first sample EEG signal 501, second sample EEG signal 502, ..., N-1 sample EEG signal 503 can be obtained as the first training sample subset corresponding to the first EEG signal classification model. Similarly, for the second EEG signal classification model, the N-1 sample EEG signal 503 is taken as the verification EEG signal corresponding to the second EEG signal classification model. Therefore, the first sample EEG signal 501, second sample EEG signal 502, ..., N-2 sample EEG signal and the Nth sample EEG signal 504 can be obtained as the second training sample subset corresponding to the first EEG signal classification model. The methods for obtaining other training sample subsets are similar and will not be elaborated here.

[0121] Step 403: Based on at least two augmented datasets obtained after augmenting at least two subsets of the training samples, train at least two EEG signal classification models corresponding to each of the at least two subsets of the training samples to obtain at least two trained EEG signal classification models.

[0122] The at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the probability distributions of the first EEG signal corresponding to the at least two EEG signal classification models, and to determine the type of motor imagery corresponding to the first EEG signal based on the probability distributions of the first EEG signal corresponding to the at least two EEG signal classification models.

[0123] Before training at least two EEG signal classification models using at least two training sample subsets, data augmentation can be performed to expand the sample EEG signals in at least two training sample subsets, increasing the number of samples in each training sample subset and improving the training effect of the EEG signal classification model.

[0124] In one possible implementation, the first training sample set is obtained; based on the first training sample set, a first training sample subset is obtained, consisting of sample EEG signals other than those corresponding to the first EEG signal classification model; the first training sample subset includes the first sample EEG signal; based on the first training sample subset, data augmentation is performed to obtain a first augmented dataset; the first augmented dataset includes the first sample EEG signal, the first augmented signal corresponding to the first sample EEG signal, and the motor imagery type corresponding to the first sample EEG signal; based on the first augmented dataset, the first EEG signal classification model is trained.

[0125] When training the first EEG signal classification model among at least two EEG signal classification models, a subset of training samples corresponding to the first EEG signal classification model can be determined first. Data augmentation can then be performed on this subset to obtain a first augmented dataset. This first augmented dataset includes the first sample EEG signal, the augmented signal corresponding to the first sample EEG signal, and the motor imagery type corresponding to the first sample EEG signal. At this point, the first sample EEG signal can be used as a sample, and the motor imagery type corresponding to the first sample EEG signal can be used as a label to train the first EEG signal classification model. Alternatively, the first augmented signal corresponding to the first sample EEG signal can be used as a sample, and the motor imagery type corresponding to the first sample EEG signal can be used as a label to train the first EEG signal classification model.

[0126] In one possible implementation, an augmentation factor corresponding to the first training sample subset is obtained, which is used to indicate the scaling ratio of the samples in the first training sample subset; based on the augmentation factor corresponding to the first training sample subset, the EEG signal of the first sample is scaled to obtain a first augmented signal corresponding to the EEG signal of the first sample; based on the EEG signal of the first sample, the first augmented signal, and the motor imagery type corresponding to the EEG signal of the first sample, the first augmented dataset is obtained.

[0127] When augmenting the first training sample subset, it is necessary to obtain the augmentation factor corresponding to the first training sample subset, and according to the augmentation factor, to scale each sample in the first training sample subset to obtain the augmented data corresponding to the EEG signals of each sample in the first training sample subset. The type of motor imagery corresponding to the augmented data obtained by scaling the sample EEG signal is the same as the type of motor imagery corresponding to the sample EEG signal. At this time, the augmented dataset contains the sample EEG signal before scaling, the augmented data obtained by scaling, and the type of motor imagery corresponding to the sample EEG signal.

[0128] In one possible implementation of this application embodiment, the first training sample subset can be augmented once to obtain an augmented dataset corresponding to the first training sample subset. At this time, the number of samples in the augmented dataset is twice that in the first training sample subset. The first training sample subset can be augmented N times to obtain an augmented dataset corresponding to the first training sample subset. At this time, the number of samples in the augmented dataset is N times that in the first training sample subset.

[0129] This scheme uses data duplication and scaling techniques to expand EEG data samples. First, the EEG data samples used for training are duplicated to obtain an expanded dataset identical to the original training data, denoted as dataset1. Each EEG sample in the training dataset is multiplied by a scaling factor α=0.9 to obtain the expanded dataset dataset2. Each EEG sample in the training dataset is multiplied by a scaling factor α=1.1 to obtain the expanded dataset dataset3. This scheme uses alpha as the scaling factor and sets the scaling factor to 0.9 and 1.1. Other scaling factor values ​​can also be used.

[0130] In one possible implementation, the first EEG signal classification model includes a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer. Based on the first sample EEG signal, the first channel attention weighting module processes the signal to obtain a first sample weighted feature map. Based on the first sample weighted feature map, the first temporal convolutional layer processes the signal to obtain a first sample temporal feature map. The first temporal convolutional layer is used to extract the temporal features of the EEG signal. Based on the first sample temporal feature map, the first spatial convolutional layer processes the signal to obtain... A first sample spatial feature map is obtained; the first spatial convolutional layer is used to extract spatial features of different regions of the head of the object corresponding to the EEG signal; based on the first sample spatial feature map, data processing is performed through the first activation layer to obtain a first sample activation feature map; based on the first sample activation feature map, data processing is performed through the first fully connected layer to obtain the probability distribution corresponding to the first sample EEG signal output by the first EEG signal classification model; based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal, the first EEG signal classification model is trained.

[0131] In one possible implementation, based on the first sample activation feature map, data processing is performed through the first fully connected layer to obtain the feature vector corresponding to the first sample EEG signal. Based on the feature vector corresponding to the first sample EEG signal, the probability distribution corresponding to the first sample EEG signal output by the first EEG signal classification model is obtained.

[0132] After the first fully connected layer processes the activation feature map of the first sample, it can output the feature vector corresponding to the EEG signal of the first sample. Then, the feature vector is normalized to obtain a normalized feature vector whose sum of the values ​​of each dimension is one. The value of each dimension of the normalized feature vector represents the probability of a certain type of motor imagery. Therefore, the values ​​of each dimension of the normalized feature vector together constitute the probability distribution corresponding to the EEG signal of the first sample.

[0133] Please refer to Figure 6 This illustrates a schematic diagram of an electroencephalogram (EEG) signal classification model involved in an embodiment of this application. Figure 6 As shown, for the input first sample EEG signal 600, which contains sample electrode signals corresponding to at least two electrodes, the channel weights corresponding to each electrode of the first sample EEG signal can be obtained through the channel attention weighting module 601. Based on the channel weights corresponding to each electrode, the sample electrode signals corresponding to each electrode are weighted to obtain a weighted first sample weighted feature map. This weighted first sample weighted feature map is processed by the first temporal convolutional layer 602 to obtain a first sample temporal feature map. Figure 6 As shown in section 602, the arrangement of each convolutional kernel in the first temporal convolutional layer is consistent with the acquisition timing of the signals in the sample electrode signal. That is, when extracting features from the sample electrode signal using the convolutional kernels of the first temporal convolutional layer, features from different time points in each electrode signal can be fused to obtain a first sample temporal feature map containing the temporal features of each electrode signal. After obtaining the first sample temporal feature map, it is further processed by the first spatial convolutional layer 603. Figure 6 As shown in section 603, each convolutional kernel of the first spatial convolutional layer 603 can simultaneously extract features from multiple electrode signals. The extracted features simultaneously contain features from multiple electrode signals. Since each electrode signal is collected based on different spatial regions of the target object's head, the first sample spatial feature map extracted by the first spatial convolutional layer 603 incorporates the spatial features of each electrode signal. After obtaining the first sample spatial feature map, a squared activation layer 604 can be used to square each element in the extracted first sample spatial feature map. After passing through a pooling layer 605 and a logarithmic activation layer 606, a first sample activation feature map is obtained. At this time, the first sample activation feature map is processed by a fully connected layer 607 to obtain the feature vector corresponding to the first sample EEG signal, and the probability distribution of motor imagery corresponding to the first sample EEG signal is obtained based on the feature vector.

[0134] This scheme designs an EEG decoding model based on channel attention, according to the temporal, spatial, and frequency characteristics of the input EEG signal. Figure 3The decoding model includes: a channel attention layer, a temporal convolutional layer, a spatial convolutional layer, a normalization layer, a squared activation layer, an average pooling layer, a log activation layer, and a fully connected layer. The input signal size is 350×40 (40 is the number of electrodes, and 350 is the time length). The first layer is the channel attention layer, which generates a channel attention map based on the input signal. The attention weights are 1×40, generated by a fully connected layer with 5 hidden nodes. The second layer is the temporal convolutional layer, with a kernel size of 25×1, a stride of 1, and 20 convolutional channels. The third layer is the spatial convolutional layer, with a kernel size of 40×1, a stride of 1, and 20 convolutional channels. The fourth layer is the batch normalization layer. Normalization is used to speed up model convergence; the fifth layer is a squared activation layer, which squares each element in the feature map to enhance the expressive power of non-linear features; the sixth layer is an average pooling layer (kernel size 35×1, stride 15×1), which compresses the feature map and performs non-linear mapping on the feature map through a Log activation layer, followed by a dropout layer to suppress overfitting, with a dropout rate of 0.5; the last layer is a fully connected layer, which fuses deep features, and its output node is 2, corresponding to the number of categories in EEG classification.

[0135] Figure 7 A schematic diagram of the channel attention mechanism involved in an embodiment of this application is shown. Figure 7 As shown, for a feature map 701 with C channels and size W×H, mean pooling is first performed on each feature map to obtain the mean of C feature maps. This mean is then mapped through a fully connected layer to form a channel attention value. The activation function of the fully connected layer is the sigmoid function. Finally, the channel attention value is multiplied by the corresponding channel feature map to form the channel attention feature map 702.

[0136] In this embodiment, the channel attention mechanism can be applied to each electrode channel pair corresponding to the first sample EEG signal. That is, the image features corresponding to each electrode channel in the first sample EEG signal are processed by the first channel attention weighting module to obtain the weights corresponding to each electrode channel in the first sample EEG signal. Based on the weights corresponding to each electrode channel, the image feature values ​​of each electrode channel in the first sample EEG signal corresponding to each electrode channel are weighted to obtain a weighted feature map.

[0137] In one possible implementation, a loss function value corresponding to the first sample EEG signal is obtained based on the probability distribution and the type of motor imagery corresponding to the first sample EEG signal, and the classification model of the first EEG signal is trained based on the loss function value corresponding to the first sample EEG signal.

[0138] In one possible implementation, a first loss function value corresponding to the first sample EEG signal is obtained based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal; a second loss function value corresponding to the first sample EEG signal is obtained based on the feature vector corresponding to the first sample EEG signal and the feature vector corresponding to the motor imagery type; and a loss function value corresponding to the first sample EEG signal is obtained based on the first loss function value and the second loss function value.

[0139] The first loss function value is a multi-class cross-entropy loss function value obtained based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal; the second loss function value is a center loss function value obtained based on the feature vector corresponding to the first sample EEG signal and the center vector corresponding to the motor imagery type corresponding to the first sample EEG signal.

[0140] In one possible implementation, the center vector corresponding to the motor imagery type of the first sample EEG signal is obtained based on the feature vectors corresponding to all sample EEG signals of that motor imagery type.

[0141] In one possible implementation, the feature vectors of all sample EEG signals corresponding to the motor imagery type corresponding to the first sample EEG signal are obtained, and the feature vectors of all sample EEG signals corresponding to the motor imagery type corresponding to the first sample EEG signal are averaged to obtain the center vector corresponding to the motor imagery type corresponding to the first sample EEG signal.

[0142] This technical solution includes three loss functions: the classifier loss function (used to obtain the first loss function value) and the Center Loss loss function (used to obtain the second loss function value).

[0143] The classifier loss function is shown below:

[0144]

[0145] in, and These are the predicted probabilities and true labels of the training data, respectively. Represents the cross-entropy loss function. Represents the overall network parameters. It is the expected value corresponding to the training data.

[0146] To improve the separability of the features extracted by the model, this application calculates the center loss based on the flattened features and their corresponding categories. The center loss function is shown below:

[0147]

[0148] in, and These are the flattened features and the feature centers of the corresponding categories.

[0149] The overall loss function of this technical solution is:

[0150]

[0151] in, To balance classification loss and Center Loss Hyperparameters.

[0152] This application embodiment can use Adam-based gradient descent to solve for the parameters of the neural network model, and use Xavier to initialize the model parameters. During the solution process, the EEG signal and corresponding label of each subject are fed into the network for learning, and model optimization is completed through error backpropagation.

[0153] In one possible implementation, in response to at least two EEG signal classification models including a first EEG signal classification model, and after the first EEG signal classification model has been trained for a specified number of rounds, the first EEG signal classification model is verified by a verification EEG signal corresponding to the first EEG signal classification model to obtain the accuracy of the first EEG signal classification model. In response to the first EEG signal classification model having an accuracy greater than an accuracy threshold, the first EEG signal classification model is determined as the trained first EEG signal classification model.

[0154] In one possible implementation, the verification EEG signal is input into the first EEG signal classification model to obtain the probability distribution corresponding to the verification EEG signal; based on the probability distribution corresponding to the verification EEG signal and the type of motor imagery corresponding to the verification EEG signal, the accuracy of the first EEG signal classification model is determined.

[0155] The accuracy of the verification EEG signal can be defined as the probability value corresponding to the motor imagery type within the probability distribution of the verification EEG signal. A higher probability value for the motor imagery type within the probability distribution indicates a more accurate prediction of the type of the verification EEG signal by the first EEG signal classification model. When this probability value (accuracy) exceeds an accuracy threshold, the first EEG signal classification model can be considered successfully trained.

[0156] In one possible implementation, in response to the fact that the accuracy of the first EEG signal classification model is less than the accuracy threshold, the first EEG signal classification model is retrained based on the training sample subset corresponding to the first EEG signal classification model. After a specified number of retraining rounds, the first EEG signal classification model is then verified by the verification EEG signal corresponding to the first EEG signal classification model.

[0157] Step 404: Obtain the first EEG signal.

[0158] In one possible implementation, the first EEG signal includes at least two electrode signals.

[0159] At least two of the first EEG signals can be EEG signals generated by the head of the target object during motor imagery, acquired through electrodes of the brain-computer interface via a terminal device with a brain-computer interface. The number of these electrode signals is the same as the number of electrodes corresponding to the brain-computer interface; that is, the brain-computer interface can acquire EEG signals generated by different spatial regions of the head of the same target object during motor imagery through different electrodes.

[0160] In one possible implementation, the brain-computer interface acquires electroencephalogram (EEG) signals generated in different regions of the head of the target object through electrodes connected to the target object, and the EEG signals corresponding to each electrode are transmitted to the terminal device corresponding to the brain-computer interface through a transmission line.

[0161] In one possible implementation, raw EEG signals generated by the head of the target object during motor imagery are acquired based on the electrodes of the brain-computer interface; the raw EEG signals are then filtered by a bandpass filter to obtain the first EEG signal.

[0162] Since the raw EEG signals obtained through the electrodes of the brain-computer interface contain a lot of noise interference, it is necessary to first filter the raw EEG signals through a bandpass filter to reduce the influence of irrelevant noise on the EEG signals.

[0163] Each raw EEG signal was subjected to a 3-38Hz bandpass filter to remove the effects of irrelevant physiological noise such as eye movements and power frequency interference (i.e., interference caused by the power system, usually 50Hz) on the EEG signal.

[0164] Step 405: The first EEG signal is processed by at least two EEG signal classification models to obtain the probability distribution of motor imagery output by at least two EEG signal classification models.

[0165] Among them, the probability distribution of motor imagery output by at least two of the EEG signal classification models is the probability distribution of motor imagery output by at least two of the EEG signal classification models after the first EEG signal is input into at least two of the EEG signal classification models respectively. That is, the probability distributions output by at least two of the EEG signal classification models are used to indicate the classification results of motor imagery corresponding to at least two of the EEG signal classification models after judging the first EEG signal by at least two of the EEG signal classification models respectively.

[0166] In one possible implementation, in response to at least two EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model includes a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer. Based on the first EEG signal, the first channel attention weighting module processes the signal to obtain a first weighted feature map; based on the first weighted feature map, the first temporal convolutional layer processes the signal to obtain a first temporal feature map; the first temporal convolutional layer is used to extract temporal features of the EEG signal; based on the first temporal feature map, the first spatial convolutional layer processes the signal to obtain a first spatial feature map; the first spatial convolutional layer is used to extract spatial features of different regions of the head corresponding to the EEG signal; based on the first spatial feature map, the first activation layer processes the signal to obtain a first activation feature map; based on the first activation feature map, the first fully connected layer processes the signal to obtain the probability distribution corresponding to the first EEG signal output by the first EEG signal classification model.

[0167] In one possible implementation, based on the first activation feature map, data processing is performed through the first fully connected layer to obtain the feature vector corresponding to the first EEG signal, and based on the feature vector corresponding to the first EEG signal, the probability distribution corresponding to the first EEG signal is obtained.

[0168] After the first fully connected layer processes the data of the first activation feature map, the first fully connected layer can output the feature vector corresponding to the first EEG signal. At this time, the feature vector is normalized to obtain a normalized feature vector in which the sum of the values ​​of each dimension is one. The value of each dimension of the normalized feature vector represents the probability of a certain type of motor imagery. Therefore, the value of each dimension of the normalized feature vector together constitutes the probability distribution corresponding to the first EEG signal.

[0169] Step 406: Based on the probability distribution of motor imagery output by at least two EEG signal classification models, determine the type of motor imagery corresponding to the first EEG signal.

[0170] In one possible implementation, a motor imagery probability distribution corresponding to the first EEG signal is obtained based on the motor imagery probability distributions output by at least two EEG signal classification models; the motor imagery probability distribution corresponding to the first EEG signal contains probability values ​​corresponding to each type of motor imagery; and the type of motor imagery corresponding to the first EEG signal is determined based on the motor imagery probability distribution corresponding to the first EEG signal.

[0171] The probability distribution of motor imagery corresponding to the first EEG signal includes probability values ​​corresponding to each type of motor imagery. These probability values ​​are used to indicate the probability of the first EEG signal corresponding to each type of motor imagery.

[0172] In one possible implementation, the highest probability value in the probability distribution of motor imagery corresponding to the first EEG signal is determined as the type of motor imagery corresponding to the first EEG signal.

[0173] In one possible implementation, in response to the probability distribution of motor imagery corresponding to the first EEG signal, if the maximum probability value is greater than a probability threshold, the type of motor imagery corresponding to the maximum probability value is determined as the type of motor imagery corresponding to the first EEG signal.

[0174] For example, when the maximum probability value in the probability distribution of motor imagery corresponding to the first EEG signal is 0.4, and the probability threshold is 0.5, the maximum probability value is less than the probability threshold. Therefore, there is no probability value greater than the probability threshold in the probability distribution of motor imagery corresponding to the first EEG signal, so the type of motor imagery corresponding to the first EEG signal can be determined as unrecognizable. When the maximum probability value in the probability distribution of motor imagery corresponding to the first EEG signal is 0.7, and the probability threshold is 0.5, the type of motor imagery corresponding to the maximum probability value of 0.7 is determined as the type of motor imagery corresponding to the first EEG signal.

[0175] In one possible implementation, the probability distributions of motor imagery output by at least two EEG signal classification models are merged based on the type of motor imagery to obtain the probability distribution of motor imagery.

[0176] In one possible implementation, the probability distribution of motor imagery output by the EEG signal classification model contains probability values ​​corresponding to each type of motor imagery. The probability values ​​corresponding to each type of motor imagery in the probability distributions of motor imagery output by at least two EEG signal classification models are weighted and summed based on the type of motor imagery to obtain the motor probability value of the first EEG signal corresponding to each type of motor imagery. Based on the motor probability value of the first EEG signal corresponding to each type of motor imagery, the probability distribution of motor imagery is obtained.

[0177] In one possible implementation, the accuracy of each EEG signal classification model is obtained; the probability values ​​corresponding to each type of motor imagery in the motor imagery probability distribution output by at least two EEG signal classification models are weighted and summed based on the accuracy of each EEG signal classification model and the type of motor imagery to obtain the motor probability value of the first EEG signal corresponding to each type of motor imagery.

[0178] The accuracy of each EEG signal classification model is used to indicate the weight of each type of motor imagery in the probability distribution of motor imagery output by each EEG signal classification model during the weighted summation process.

[0179] In one possible implementation, each EEG classification model is validated based on the validation EEG signals corresponding to it, thereby determining the accuracy of each model. Specifically, when validating with validation EEG signals, the EEG classification model with higher accuracy typically exhibits better training performance. Therefore, the probability distribution output by the more accurate EEG classification model is more reliable and can have a larger weighting in the weighted summation process.

[0180] In one possible implementation, the probability values ​​corresponding to each type of motor imagery in the motor imagery probability distribution output by at least two of the EEG signal classification models are averaged based on the motor imagery type to obtain the motor probability value corresponding to each type of motor imagery for the first EEG signal.

[0181] Please refer to Figure 8 This illustrates a schematic diagram of the merging of motion imagination probability distributions related to this application. For example... Figure 8As shown, taking the EEG signal classification model, which includes a first EEG signal classification model and a second EEG signal classification model, as an example, the first motor imagery probability distribution 801 is the probability distribution of the first EEG signal classification model based on the output of the first EEG signal, and the second motor imagery probability distribution 802 is the probability distribution of the second EEG signal classification model based on the output of the first EEG signal. In the first motor imagery probability distribution 801, (0.1, 0.3, 0.5, 0.1) respectively indicate the probability of the first EEG signal corresponding to the four types of motor imagery (A, B, C, D); in the second motor imagery probability distribution 802, (0.2, 0.3, 0.3, 0.2) respectively indicate the probability of the first EEG signal corresponding to the four types of motor imagery (A, B, C, D). At this point, the probabilities corresponding to the same type in the first motor imagery probability distribution 801 and the second motor imagery probability distribution 802 can be averaged to obtain the motor imagery probability distribution 803 corresponding to the first EEG signal.

[0182] In one possible implementation, the type of motor imagery corresponding to the largest probability value of the motor imagery probability distribution output by at least two EEG signal classification models is determined as the type of motor imagery corresponding to the first EEG signal.

[0183] Once the probability distributions of motor imagery output by at least two EEG signal classification models are obtained, the type of motor imagery corresponding to the largest probability value among all the probability values ​​contained in the probability distributions of motor imagery output by at least two EEG signal classification models can be determined as the type of motor imagery corresponding to the first EEG signal.

[0184] When using n optimized EEG signal classification models to classify new EEG data samples, the samples are first input into each sub-model for prediction, resulting in n predicted probabilities. When these n predicted probabilities are summed and averaged to obtain the final predicted probability for classification, this ensemble method is called "average ensemble." When the category corresponding to the highest probability among the n predicted probabilities is used as the final classification category, this ensemble method is called "maximum ensemble." In practical applications, the choice between these two ensemble methods is determined based on the specific subject signals.

[0185] In summary, the scheme described in this application trains multiple EEG signal classification models by augmenting different training subsets of the same training sample set. These augmented data sets are then used to process EEG signals, yielding probability distributions output by each model. The corresponding motor imagery type is determined based on these probability distributions. This scheme, by taking different subsets of a training sample set and augmenting them to train at least two EEG signal classification models, and then using these models to classify EEG signals to determine their corresponding motor imagery type, improves the training effect of the model with a limited number of training samples by dividing the sample set into multiple subsets and then augmenting it. Furthermore, by considering the outputs of multiple EEG signal classification models simultaneously during EEG signal classification, the accuracy of EEG signal classification is improved.

[0186] Figure 9 This is a flowchart illustrating an electroencephalogram (EEG) signal classification according to an exemplary embodiment. In this embodiment, the EEG signal classification includes a model training process and a model application process. The model training and application processes can be jointly executed by a model training device 900 and a model application device (i.e., a signal processing device) 910, such as... Figure 9 As shown, the model training and application process is as follows:

[0187] In the model training device 900, the model training device acquires a first training sample set 901 for training various EEG signal classification models. The first training sample set 901 contains N sample EEG signals for training and the motor imagery types corresponding to the N sample EEG signals.

[0188] Before the model training device trains each EEG signal classification model, it is necessary to determine the training set corresponding to each EEG signal classification model. Taking the training process of the first EEG signal classification model as an example, when the model training device needs to train the first EEG signal classification model, it can first select the verification EEG signal corresponding to the first EEG signal classification model from the sample EEG signals in the first training sample set 901, so that the first EEG signal classification model can be verified using the verification EEG signal after training.

[0189] The training set (i.e. the first data subset) corresponding to the first EEG signal classification model can be constructed based on all sample EEG signals other than the verification EEG signal corresponding to the first EEG signal classification model. Taking the verification EEG signal corresponding to the first EEG signal classification model as the Nth sample EEG signal as an example, the first data subset contains all sample EEG signals in the first training sample set except for the Nth sample EEG signal, as well as the motor imagery type corresponding to each sample EEG signal.

[0190] Since the sample data for motor imagery is usually limited, it is necessary to increase the number of training samples for the model through data augmentation. This can be achieved by performing data augmentation on the training set corresponding to each EEG signal classification model to obtain augmented datasets for each EEG signal classification model. Each EEG signal classification model can then be trained using the augmented datasets of the training sets corresponding to each EEG signal classification model to obtain the trained EEG signal classification models.

[0191] After each EEG signal classification model has been trained for a predetermined number of rounds, the model training device 900 can verify each EEG signal classification model using the verification EEG signals corresponding to each EEG signal classification model, and send the verified EEG signal classification model to the model application device to achieve the classification processing of EEG signals.

[0192] In the model application device 910, for the acquired first EEG signal 911, each EEG signal classification model (e.g., first EEG signal classification model, second EEG signal classification model, ... Nth EEG signal classification model) is input to obtain the probability distribution output by each EEG signal classification model. Based on the probability distribution output by each EEG signal classification model, the motor imagery type 912 corresponding to the first EEG signal is obtained through the above-mentioned average integration or maximum integration method.

[0193] Figure 10 This is a structural block diagram illustrating an electroencephalogram (EEG) signal classification device according to an exemplary embodiment. This EEG signal classification device can achieve [the following is a description of the device's capabilities]: Figure 2 or Figure 4 The method provided in the illustrated embodiment includes all or part of the steps, and the EEG signal classification device includes:

[0194] The EEG signal acquisition module 1001 is used to acquire the first EEG signal;

[0195] The probability distribution acquisition module 1002 is used to process the first EEG signal using at least two EEG signal classification models to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively. The EEG signal classification models are machine learning models trained on an augmented dataset obtained by augmenting a subset of training samples. The training sample subset includes sample EEG signals in a first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models. The first training sample set includes at least two of the sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification models. The verification EEG signals corresponding to the at least two EEG signal classification models in the first training sample set are different.

[0196] The motor imagery type acquisition module 1003 is used to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distributions output by at least two EEG signal classification models.

[0197] In one possible implementation, the motion imagination type acquisition module 1003 includes:

[0198] The EEG probability distribution acquisition submodule is used to acquire the motor imagery probability distribution corresponding to the first EEG signal based on the motor imagery probability distributions output by at least two EEG signal classification models respectively; the probability distribution corresponding to the first EEG signal contains probability values ​​corresponding to each type of motor imagery.

[0199] The motor imagery type acquisition submodule is used to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distribution corresponding to the first EEG signal.

[0200] In one possible implementation, the EEG probability distribution acquisition submodule is further used for,

[0201] The probability distributions of motor imagery output by at least two of the EEG signal classification models are merged based on the type of motor imagery to obtain the probability distribution of motor imagery corresponding to the first EEG signal.

[0202] In one possible implementation, the probability distribution of motor imagery output by the EEG signal classification model includes probability values ​​corresponding to each type of motor imagery.

[0203] The EEG probability distribution acquisition submodule is also used for,

[0204] The probability distributions of motor imagery output by at least two of the EEG signal classification models are weighted and summed based on the type of motor imagery to obtain the probability distribution of motor imagery corresponding to the first EEG signal.

[0205] In one possible implementation, the motion imagination type acquisition module 1003 is further used for,

[0206] The type of motor imagery corresponding to the highest probability value in the motor imagery probability distributions output by at least two of the aforementioned EEG signal classification models is determined as the type of motor imagery corresponding to the first EEG signal.

[0207] In one possible implementation, in response to at least two of the EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model including a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer;

[0208] The probability distribution acquisition module 1002 includes:

[0209] The first weighting submodule is used to process the first EEG signal through the first channel attention weighting module to obtain a first weighted feature map.

[0210] The first time extraction submodule is used to process the first weighted feature map through the first time convolutional layer to obtain the first time feature map; the first time convolutional layer is used to extract the temporal features of the EEG signal.

[0211] The first spatial extraction submodule is used to process the first temporal feature map through the first spatial convolutional layer to obtain the first spatial feature map; the first spatial convolutional layer is used to extract the spatial features of different regions of the head of the object corresponding to the EEG signal.

[0212] The first activation submodule is used to process data through the first activation layer based on the first spatial feature map to obtain the first activation feature map.

[0213] The first probability distribution acquisition submodule is used to obtain the probability distribution of motor imagery corresponding to the first EEG signal output by the first EEG signal classification model by processing the data through the first fully connected layer based on the first activation feature map.

[0214] In one possible implementation, the device further includes:

[0215] The first sample set acquisition module is used to acquire the first training sample set;

[0216] The first sample subset acquisition module is used to acquire a first training sample subset based on the sample EEG signals in the first training sample set, excluding the verification EEG signals corresponding to the first EEG signal classification model; the first training sample subset includes the first sample EEG signals.

[0217] The data augmentation module is used to augment the data based on the first training sample subset to obtain a first augmented dataset; the first augmented dataset includes the first sample EEG signal, the first augmented signal corresponding to the first sample EEG signal, and the motor imagery type corresponding to the first sample EEG signal.

[0218] The first training module is used to train the first EEG signal classification model based on the first augmented dataset.

[0219] In one possible implementation, the data augmentation module includes:

[0220] An augmentation factor acquisition submodule is used to acquire the augmentation factor corresponding to the first training sample subset, wherein the augmentation factor is used to indicate the scaling ratio of the samples in the first training sample subset.

[0221] The data augmentation submodule is used to scale the EEG signal of the first sample based on the augmentation factor corresponding to the first training sample subset to obtain the first augmented signal corresponding to the EEG signal of the first sample.

[0222] In one possible implementation, the device further includes:

[0223] The verification signal acquisition module is used to determine the verification EEG signal corresponding to the first EEG signal classification model from at least two sample EEG signals in the first training sample set, based on the first EEG signal classification model.

[0224] In summary, the scheme described in this application trains multiple EEG signal classification models by augmenting different training subsets of the same training sample set. These augmented data sets are then used to process EEG signals, yielding probability distributions output by each model. The corresponding motor imagery type is determined based on these probability distributions. This scheme, by taking different subsets of a training sample set and augmenting them to train at least two EEG signal classification models, and then using these models to classify EEG signals to determine their corresponding motor imagery type, improves the training effect of the model with a limited number of training samples by dividing the sample set into multiple subsets and then augmenting it. Furthermore, by considering the outputs of multiple EEG signal classification models simultaneously during EEG signal classification, the accuracy of EEG signal classification is improved.

[0225] Figure 11 This is a structural block diagram illustrating an electroencephalogram (EEG) signal classification device according to an exemplary embodiment. This EEG signal classification device can achieve [the following is a description of the device's capabilities]: Figure 3 or Figure 4 The method provided in the illustrated embodiment includes all or part of the steps, and the EEG signal classification device includes:

[0226] The training subset acquisition module 1101 is used to acquire a first training sample set; the first training sample set includes at least two sample EEG signals and at least two motor imagery types corresponding to the sample EEG signals.

[0227] The training sample subset acquisition module 1102 is used to acquire training sample subsets corresponding to at least two EEG signal classification models based on the first training sample set and at least two EEG signal classification models. The training sample subsets include the sample EEG signals in the first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models. The sample EEG signals corresponding to the at least two EEG signal classification models in the first training sample set are different. The first training sample set includes at least two sample EEG signals and motor imagery types corresponding to the at least two sample EEG signals. The verification EEG signals are sample EEG signals used to verify the EEG signal classification models. The verification EEG signals corresponding to the at least two EEG signal classification models in the first training sample set are different.

[0228] The model training module 1103 is used to train at least two EEG signal classification models corresponding to at least two training sample subsets based on at least two augmented datasets obtained after data augmentation of at least two training sample subsets, to obtain at least two trained EEG signal classification models.

[0229] The at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the motor imagery probability distributions output by the at least two EEG signal classification models respectively, and to determine the motor imagery type corresponding to the first EEG signal based on the motor imagery probability distributions output by the at least two EEG signal classification models respectively.

[0230] In one possible implementation, the training sample subset acquisition module 1102 includes:

[0231] The verification signal acquisition submodule is used to determine the verification EEG signal corresponding to the at least two EEG signal classification models respectively in at least two sample EEG signals in the first training sample set, based on at least two EEG signal classification models.

[0232] The training sample subset acquisition submodule 1102 is used to acquire training sample subsets corresponding to at least two of the EEG signal classification models based on the verification EEG signals corresponding to at least two of the EEG signal classification models, and the first training sample set.

[0233] In summary, the scheme described in this application trains multiple EEG signal classification models by augmenting different training subsets of the same training sample set. These augmented data sets are then used to process EEG signals, yielding probability distributions output by each model. The corresponding motor imagery type is determined based on these probability distributions. This scheme, by taking different subsets of a training sample set and augmenting them to train at least two EEG signal classification models, and then using these models to classify EEG signals to determine their corresponding motor imagery type, improves the training effect of the model with a limited number of training samples by dividing the sample set into multiple subsets and then augmenting it. Furthermore, by considering the outputs of multiple EEG signal classification models simultaneously during EEG signal classification, the accuracy of EEG signal classification is improved.

[0234] Figure 12This is a schematic diagram illustrating the structure of a computer device according to an exemplary embodiment. The computer device can be implemented as a model training device and / or signal processing device in the various method embodiments described above. The computer device 1200 includes a central processing unit (CPU) 1201, a system memory 1204 including random access memory (RAM) 1202 and read-only memory (ROM) 1203, and a system bus 1205 connecting the system memory 1204 and the central processing unit 1201. The computer device 1200 also includes a basic input / output system 1206 to facilitate information transfer between various devices within the computer, and a mass storage device 1207 for storing an operating system 1213, application programs 1214, and other program modules 1215.

[0235] The mass storage device 1207 is connected to the central processing unit 1201 via a mass storage controller (not shown) connected to the system bus 1205. The mass storage device 1207 and its associated computer-readable media provide non-volatile storage for the computer device 1200. That is, the mass storage device 1207 may include computer-readable media (not shown), such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0236] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include RAM, ROM, flash memory or other solid-state storage technologies, CD-ROM, or other optical storage, magnetic tape cassettes, magnetic tape, disk storage, or other magnetic storage devices. Of course, those skilled in the art will recognize that the computer storage media are not limited to the above-mentioned types. The system memory 1204 and the mass storage device 1207 described above can be collectively referred to as memory.

[0237] Computer device 1200 can be connected to the Internet or other network devices via network interface unit 1211 connected to the system bus 1205.

[0238] The memory also includes one or more programs, which are stored in the memory, and the central processing unit 1201 implements these programs by executing them. Figure 2 , Figure 3 or Figure 4 All or part of the steps of the method shown.

[0239] In exemplary embodiments, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory including a computer program (instructions) that can be executed by a processor of a computer device to perform the methods shown in the various embodiments of this application. For example, the non-transitory computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage device, etc.

[0240] In an exemplary embodiment, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods shown in the various embodiments described above.

[0241] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the claims.

[0242] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method for classifying electroencephalogram (EEG) signals, characterized in that, The method includes: Acquire a first electroencephalogram (EEG) signal, which includes at least two electrode signals; The first EEG signal is processed by at least two EEG signal classification models to obtain the probability distribution of motor imagery output by each of the at least two EEG signal classification models. Responding to the at least two EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model includes a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer. Based on the first EEG signal, the first channel attention weighting module performs mean pooling on the feature maps corresponding to each electrode channel in the first EEG signal to obtain C feature map mean values. These C feature map mean values ​​are then mapped through a fully connected layer to form the probability distribution of motor imagery in the first EEG signal. The channel weights corresponding to each electrode are used to weight the channel feature maps corresponding to each electrode, resulting in a first weighted feature map. Based on the first weighted feature map, a first temporal feature map is obtained through a first temporal convolutional layer. The first temporal convolutional layer is used to extract the temporal features of the EEG signal. The arrangement of each convolutional kernel in the first temporal convolutional layer is consistent with the signal acquisition time sequence in the first weighted feature map. The first temporal convolutional layer is used to fuse features from different time points in the signal from each electrode. Based on the first temporal feature map, a first spatial convolutional layer is used to obtain a first spatial feature map. The first spatial convolutional layer is used to extract EEG signals. The spatial features of different regions of the head corresponding to the signal are used to extract features from multiple electrode signals simultaneously, with each convolutional kernel of the first spatial convolutional layer. Based on the first spatial feature map, data processing is performed through the first activation layer to obtain a first activation feature map. Based on the first activation feature map, data processing is performed through the first fully connected layer to obtain the probability distribution of motor imagery corresponding to the first EEG signal, output by the first EEG signal classification model. The EEG signal classification model is a machine learning model trained on an augmented dataset obtained by augmenting a subset of training samples. The subset of training samples includes the first training sample set, except for those corresponding to the EEG signal classification model. The training sample set includes at least two sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification model. The verification EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set. The loss function value used to train the first EEG signal classification model is obtained based on a first loss function value and a second loss function value. The first loss function value is obtained based on the probability distribution corresponding to the first sample EEG signal and the corresponding motor imagery type.The second loss function value is obtained based on the feature vector corresponding to the first sample EEG signal and the center vector corresponding to the motor imagery type of the first sample EEG signal; the center vector corresponding to the motor imagery type of the first sample EEG signal is obtained based on the feature vectors corresponding to all sample EEG signals corresponding to the motor imagery type of the first sample EEG signal. The accuracy of each EEG signal classification model is obtained, which is based on the verification EEG signals corresponding to each EEG signal classification model and is obtained by verifying each EEG signal classification model. The probability values ​​corresponding to each motor imagery type in the motor imagery probability distribution output by at least two of the EEG signal classification models are weighted and summed based on the accuracy of each EEG signal classification model and the motor imagery type to obtain the motor probability value of the first EEG signal corresponding to each motor imagery type. Based on the motor probability value of the first EEG signal corresponding to each motor imagery type, the motor imagery probability distribution corresponding to the first EEG signal is obtained. Based on the motor imagery probability distribution corresponding to the first EEG signal, the motor imagery type corresponding to the first EEG signal is determined.

2. The method according to claim 1, characterized in that, The method further includes: The type of motor imagery corresponding to the highest probability value in the motor imagery probability distributions output by at least two of the aforementioned EEG signal classification models is determined as the type of motor imagery corresponding to the first EEG signal.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Obtain the first training sample set; Based on the first training sample set, a first training sample subset is obtained from the sample EEG signals other than the verification EEG signals corresponding to the first EEG signal classification model; the first training sample subset includes the first sample EEG signals. Based on the first training sample subset, data augmentation is performed to obtain a first augmented dataset; the first augmented dataset includes the first sample EEG signal, the first augmented signal corresponding to the first sample EEG signal, and the motor imagery type corresponding to the first sample EEG signal. The first EEG signal classification model is trained based on the first augmented dataset.

4. The method according to claim 3, characterized in that, The step of augmenting the data based on the first training sample subset to obtain the first augmented dataset includes: Obtain the augmentation factor corresponding to the first training sample subset, wherein the augmentation factor is used to indicate the scaling ratio of the samples in the first training sample subset; Based on the augmentation factor corresponding to the first training sample subset, the EEG signal of the first sample is scaled up to obtain the first augmented signal corresponding to the EEG signal of the first sample.

5. The method according to claim 3, characterized in that, Before obtaining the first training sample subset based on all sample EEG signals in the first training sample set, excluding the validation EEG signals corresponding to the first EEG signal classification model, the process further includes: Based on the first EEG signal classification model, the verification EEG signal corresponding to the first EEG signal classification model is determined from at least two sample EEG signals in the first training sample set.

6. A method for classifying electroencephalogram (EEG) signals, characterized in that, The method includes: Obtain a first training sample set; the first training sample set includes at least two sample EEG signals, and at least two motor imagery types corresponding to the sample EEG signals; Based on the first training sample set and at least two EEG signal classification models, at least two training sample subsets corresponding to the EEG signal classification models are obtained respectively; the training sample subsets include sample EEG signals from the first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models; the sample EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set; the first training sample set includes at least two sample EEG signals and at least two motor imagery types corresponding to the sample EEG signals; the verification EEG signals are sample EEG signals used to verify the EEG signal classification models; the verification EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set. At least two augmented datasets are obtained by augmenting data based on at least two subsets of the training samples. At least two EEG signal classification models corresponding to each of the at least two subsets of the training samples are trained to obtain at least two trained EEG signal classification models. In response to the at least two EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model includes a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer. Based on a first sample EEG signal in the augmented dataset corresponding to the first EEG signal classification model, the first channel attention weighting module is used to apply attention weighting to each electrode in the first sample EEG signal. The feature maps corresponding to each channel are subjected to mean pooling to obtain C mean values. These C mean values ​​are then mapped through a fully connected layer to form channel weights corresponding to each electrode in the first sample EEG signal. Based on these channel weights, the channel feature maps corresponding to each electrode are weighted to obtain a first sample weighted feature map. The first sample EEG signal includes signals from at least two sample electrodes. Based on the first sample weighted feature map, the signal is processed through a first temporal convolutional layer to obtain a first sample temporal feature map. The first temporal convolutional layer is used to extract the temporal features of the EEG signal. The arrangement of each convolutional kernel in the first temporal convolutional layer corresponds to the signal in the first sample weighted feature map. The acquisition timing is consistent. The first temporal convolutional layer is used to fuse features from different time points in each electrode signal. Based on the first sample temporal feature map, it is processed by the first spatial convolutional layer to obtain the first sample spatial feature map. The first spatial convolutional layer is used to extract spatial features of different regions of the head corresponding to the EEG signal. Each convolutional kernel of the first spatial convolutional layer is used to extract features from multiple electrode signals simultaneously. Based on the first sample spatial feature map, it is processed by the first activation layer to obtain the first sample activation feature map. Based on the first sample activation feature map, it is processed by the first fully connected layer to obtain the output of the first EEG signal classification model. The probability distribution corresponding to the first sample EEG signal; based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal, the first EEG signal classification model is trained; the loss function value used to train the first EEG signal classification model is obtained based on a first loss function value and a second loss function value, wherein the first loss function value is obtained based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal; and the second loss function value is obtained based on the feature vector corresponding to the first sample EEG signal and the center vector corresponding to the motor imagery type corresponding to the first sample EEG signal.The center vector corresponding to the motor imagery type of the first sample EEG signal is obtained based on the feature vectors corresponding to all sample EEG signals of the motor imagery type corresponding to the first sample EEG signal. Specifically, at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the probability distributions of motor imagery output by the at least two EEG signal classification models, and to obtain the accuracy corresponding to each EEG signal classification model. The accuracy is obtained by verifying each EEG signal classification model based on the verification EEG signals corresponding to each EEG signal classification model. The probability values ​​corresponding to each type of motor imagery in the probability distributions of motor imagery output by the at least two EEG signal classification models are weighted and summed based on the accuracy of each EEG signal classification model and the type of motor imagery to obtain the motor probability values ​​corresponding to the first EEG signal and each type of motor imagery. Based on the motor probability values ​​corresponding to the first EEG signal and each type of motor imagery, the probability distribution of motor imagery corresponding to the first EEG signal is obtained. Based on the probability distribution of motor imagery corresponding to the first EEG signal, the type of motor imagery corresponding to the first EEG signal is determined.

7. The method according to claim 6, characterized in that, The step of obtaining at least two training sample subsets corresponding to the first training sample set and at least two EEG signal classification models includes: Based on at least two of the EEG signal classification models, the verification EEG signals corresponding to the at least two EEG signal classification models are determined in the first training sample set; Based on the verification EEG signals corresponding to at least two of the EEG signal classification models, and the first training sample set, obtain training sample subsets corresponding to at least two of the EEG signal classification models.

8. A brainwave signal classification device, characterized in that, The device includes: An electroencephalogram (EEG) signal acquisition module is used to acquire a first EEG signal, wherein the first EEG signal includes at least two electrode signals; The probability distribution acquisition module is used to process the first EEG signal using at least two EEG signal classification models to obtain the probability distribution of motor imagery output by the at least two EEG signal classification models respectively; in response to the at least two EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model including a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer; based on the first EEG signal, the first channel attention weighting module performs mean pooling on the feature maps corresponding to each electrode channel in the first EEG signal to obtain C feature map mean values, and maps the C feature map mean values ​​through a fully connected layer to form the probability distribution of motor imagery. The first EEG signal is processed by weighting the channel weights corresponding to each electrode. Based on these weights, the channel feature maps corresponding to each electrode are weighted to obtain a first weighted feature map. The first weighted feature map is then processed by a first temporal convolutional layer to obtain a first temporal feature map. This first temporal convolutional layer extracts the temporal features of the EEG signal. The arrangement of each convolutional kernel in the first temporal convolutional layer corresponds to the acquisition time sequence of the signal in the first weighted feature map. The first temporal convolutional layer also fuses features from different time points in each electrode signal. Finally, the first spatial convolutional layer is used to process the first temporal feature map to obtain a first spatial feature map. To extract spatial features of different regions of the head corresponding to EEG signals, each convolutional kernel of the first spatial convolutional layer is used to simultaneously extract features from multiple electrode signals. Based on the first spatial feature map, data processing is performed through the first activation layer to obtain a first activation feature map. Based on the first activation feature map, data processing is performed through the first fully connected layer to obtain the probability distribution of motor imagery corresponding to the first EEG signal, output by the first EEG signal classification model. The EEG signal classification model is a machine learning model obtained by training a subset of training samples using an augmented dataset. The subset of training samples includes the first training sample set, except for those related to the EEG signal classification model. The training sample set includes at least two sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification model. The verification EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set. The loss function value used to train the first EEG signal classification model is obtained based on a first loss function value and a second loss function value. The first loss function value is obtained based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal.The second loss function value is obtained based on the feature vector corresponding to the first sample EEG signal and the center vector corresponding to the motor imagery type of the first sample EEG signal; the center vector corresponding to the motor imagery type of the first sample EEG signal is obtained based on the feature vectors corresponding to all sample EEG signals corresponding to the motor imagery type of the first sample EEG signal. The motor imagery type acquisition module is used to acquire the accuracy corresponding to each EEG signal classification model. The accuracy is obtained by verifying each EEG signal classification model based on the verification EEG signals corresponding to each EEG signal classification model. The module then takes the probability values ​​corresponding to each motor imagery type in the motor imagery probability distributions output by at least two of the EEG signal classification models, and performs a weighted sum based on the accuracy of each EEG signal classification model and the motor imagery type to obtain the motor probability values ​​corresponding to the first EEG signal and each motor imagery type. Based on the motor probability values ​​corresponding to the first EEG signal and each motor imagery type, the module acquires the motor imagery probability distribution corresponding to the first EEG signal. Based on the motor imagery probability distribution corresponding to the first EEG signal, the module determines the motor imagery type corresponding to the first EEG signal.

9. A brainwave signal classification device, characterized in that, The device includes: The training subset acquisition module is used to acquire a first training sample set; the first training sample set includes at least two sample EEG signals and at least two motor imagery types corresponding to the sample EEG signals; A training sample subset acquisition module is used to acquire training sample subsets corresponding to at least two EEG signal classification models, based on the first training sample set and at least two EEG signal classification models. The training sample subsets include sample EEG signals from the first training sample set, excluding the verification EEG signals corresponding to the EEG signal classification models. The sample EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set. The first training sample set includes at least two sample EEG signals and at least two corresponding motor imagery types. The verification EEG signals are sample EEG signals used to verify the EEG signal classification models. The verification EEG signals corresponding to the at least two EEG signal classification models are different in the first training sample set. The model training module is configured to obtain at least two augmented datasets by augmenting at least two subsets of the training samples, and to train at least two EEG signal classification models corresponding to each of the at least two subsets of the training samples to obtain at least two trained EEG signal classification models; in response to the at least two EEG signal classification models including a first EEG signal classification model, the first EEG signal classification model including a first channel attention weighting module, a first temporal convolutional layer, a first spatial convolutional layer, a first activation layer, and a first fully connected layer; based on the first sample EEG signal in the augmented dataset corresponding to the first EEG signal classification model, the first sample EEG signal is processed by the first channel attention weighting module. The feature maps corresponding to each electrode channel in the signal are subjected to mean pooling to obtain C feature map mean values. These C feature map mean values ​​are then mapped through a fully connected layer to form the channel weights corresponding to each electrode in the first sample EEG signal. Based on these channel weights, the channel feature maps corresponding to each electrode are weighted to obtain a first sample weighted feature map. The first sample EEG signal includes at least two sample electrode signals. Based on the first sample weighted feature map, a first temporal convolutional layer is used to process the signal to obtain a first sample temporal feature map. The arrangement of each convolutional kernel in the first temporal convolutional layer is consistent with the acquisition timing of the signals in the first sample weighted feature map. The first temporal convolutional layer is used for... Features from different time points in each electrode signal are fused; the first temporal convolutional layer is used to extract the temporal features of the EEG signal; based on the first sample temporal feature map, it is processed by the first spatial convolutional layer to obtain the first sample spatial feature map; the first spatial convolutional layer is used to extract the spatial features of different regions of the head corresponding to the EEG signal, and each convolutional kernel of the first spatial convolutional layer is used to extract features from multiple electrode signals simultaneously; based on the first sample spatial feature map, it is processed by the first activation layer to obtain the first sample activation feature map; based on the first sample activation feature map, it is processed by the first fully connected layer to obtain the first EEG signal classification model output. The probability distribution corresponding to the first sample EEG signal is obtained; based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal, the first EEG signal classification model is trained; the loss function value used to train the first EEG signal classification model is obtained based on a first loss function value and a second loss function value, wherein the first loss function value is obtained based on the probability distribution corresponding to the first sample EEG signal and the motor imagery type corresponding to the first sample EEG signal; and the second loss function value is obtained based on the feature vector corresponding to the first sample EEG signal and the center vector corresponding to the motor imagery type corresponding to the first sample EEG signal.The center vector corresponding to the motor imagery type of the first sample EEG signal is obtained based on the feature vectors corresponding to all sample EEG signals of the motor imagery type corresponding to the first sample EEG signal. The at least two trained EEG signal classification models are used to process the input first EEG signal to obtain the probability distributions of motor imagery output by the at least two EEG signal classification models, and to obtain the accuracy of each EEG signal classification model. The accuracy is obtained by verifying each EEG signal classification model based on the verification EEG signals corresponding to each EEG signal classification model. The probability values ​​corresponding to each type of motor imagery in the probability distributions of motor imagery output by the at least two EEG signal classification models are weighted and summed based on the accuracy of each EEG signal classification model and the type of motor imagery to obtain the motor probability values ​​of the first EEG signal and each type of motor imagery. Based on the motor probability values ​​of the first EEG signal and each type of motor imagery, the probability distribution of motor imagery corresponding to the first EEG signal is obtained. Based on the probability distribution of motor imagery corresponding to the first EEG signal, the type of motor imagery corresponding to the first EEG signal is determined.

10. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, at least one program, code set, or instruction set, the at least one instruction, the at least one program, the code set, or instruction set being loaded and executed by the processor to implement the EEG signal classification method as described in any one of claims 1 to 7.

11. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the EEG signal classification method as described in any one of claims 1 to 7.

Images