Brain electrical signal classification method, device, electronic equipment, medium and product

Through the teacher-student network method, using labeled and labelless data to perform multiple rounds of training under the conditions of small data sets, filtering and training labelless data, solving the problem of insufficient classification accuracy of deep learning models under small data sets, and improving the The accuracy of the EEG signal classification model is achieved and the cost of data annotation is reduced.

CN114626408BActive Publication Date: 2024-11-05TSINGHUA SHENZHEN INTERNATIONAL GRADUATE SCHOOL
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202210152848.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-18
Publication Date
2024-11-05
Estimated Expiration
2042-02-18

AI Technical Summary

Technical Problem

In the case of small data sets, the classification accuracy of the deep learning model is insufficient, and the problem of unclear features due to insufficient training in the feature extraction and analysis process.

Method used

Using the teacher-student network method, the neural network model to be trained by multiple selection of complexity sorting, using labeled and labelless data for training, and is divided into the first and second categories of labelless data, and using the teacher network model to the non-labeled data The label data is filtered and trained to gradually improve the accuracy of the student network model.

Benefits of technology

Under the condition of a small amount of labeled data and a large amount of unlabeled data, the accuracy of the EEG signal classification model is improved, the dependence on high-quality data is reduced, and the data labeling cost is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114626408B_ABST
    Figure CN114626408B_ABST
Patent Text Reader

Abstract

The present invention provides an EEG signal classification method, device, electronic device, medium and product, including: obtaining a target EEG signal to be processed; inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal; wherein the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data, and multiple selections of teacher network models and student network models from multiple neural network models to be trained sorted by complexity, and is obtained based on the student network model finally obtained by training. The present invention can obtain an EEG signal classification model with high classification accuracy, thereby obtaining more accurate classification results, and can also greatly reduce the dependence of the deep learning model on high-quality data and reduce the data annotation cost.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of signal processing technology, and in particular to an electroencephalogram signal classification method, device, electronic equipment, medium and product. Background Art

[0002] Brain waves are the result of the sum of the postsynaptic potentials of a large number of neurons in the cerebral cortex. The formation of synchronous rhythm of brain waves is related to the activity of the nonspecific projection system of the cortex and thalamus, which can be obtained by amplifying and recording the spontaneous biopotential of the brain from the scalp through precise electronic instruments. The recorded graph is the electroencephalogram (EEG). EEG signals can be divided into four basic types according to the spectrum: α wave, β wave, θ wave and δ wave. By classifying EEG signals, it can help patients with limb disabilities such as hemiplegia caused by stroke to undergo rehabilitation training or control wheelchair travel. It can also be used for education and entertainment for ordinary users. It is closely related to human life and health.

[0003] The existing EEG signal classification algorithms mainly include adaptive classifiers, matrix and tensor learning classifiers, transfer learning, and deep learning. Among them, the deep learning algorithm extracts features and trains classifiers from EEG signal data. In theory, deep learning can achieve more effective feature extraction and more accurate pattern classification of EEG, but in practice, due to the small number of EEG signal samples, the trained EEG signal classification model has poor accuracy and low accuracy in EEG signal classification.

[0004] At present, deep learning has achieved excellent performance in solving problems such as classification, clustering, and intelligent decision-making. Its main model training method is supervised learning, that is, based on the provided labeled data, systematically learn the deep meaning of the data, explore the data rules, and establish data correlation. And establish a loss function based on the label, use the back propagation principle to automatically adjust the model parameters, and establish a convergent deep learning model. In addition, with the substantial improvement of computing power and the great enrichment of data resources, deep learning technology has developed rapidly and has been deeply integrated with many industries, realizing technology-driven, greatly improving classification accuracy, shortening time, reducing production and decision-making costs, and profoundly changing the way people live.

[0005] However, deep learning technology also has obvious flaws. One of the major problems of this technology is the heavy reliance on high-quality data during model training. High-quality models often require a large amount of labeled data, and the accuracy and availability of the labels must be guaranteed. At the same time, even if there is sufficient data, if you want to further improve the accuracy of the model, you need to provide a more complex model with more layers and more parameters, which will further increase the demand for labeled data, leading to further increases in costs.

[0006] In many cases, data itself is not lacking, but data with accurate labels is very lacking. In the fields of natural language processing, medical diagnosis, etc., accurately labeling data is often a very costly and time-consuming process. It will greatly increase the training cost of the model and reduce the availability and popularity of the model. Therefore, a large amount of unlabeled data and a small amount of labeled data are scenarios we often encounter when using deep learning technology for supervised classification. In this scenario, it becomes very necessary to improve the classification accuracy and other related performance of the model without increasing the amount of labeled data. Summary of the invention

[0007] The present invention provides an EEG signal classification method, device, electronic device, medium and product, which are used to solve the problem of insufficient accuracy of supervised classification in the case of small data sets, and unclear features caused by insufficient model training during feature extraction and analysis.

[0008] The present invention provides an electroencephalogram signal classification method, comprising: obtaining a target electroencephalogram signal to be processed;

[0009] Inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal;

[0010] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0011] According to an EEG signal classification method provided by the present invention, before inputting the target EEG signal to be processed into an EEG signal classification model, the method further includes:

[0012] Step 1), selecting a teacher network model and a student network model from a set of neural network models to be trained; wherein the complexity of the teacher network model is lower than the complexity of the student network model; the neural network models in the set of neural network models to be trained are sorted in order of complexity;

[0013] Step 2), training the teacher network model using labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model;

[0014] Step 3), inputting the unlabeled EEG signal training data into the trained teacher network model, and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to the output result of the trained teacher network model; wherein the confidence of the first category of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second category of unlabeled EEG signal training data is less than the predetermined confidence threshold;

[0015] Step 4), based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data and the second type of unlabeled EEG signal training data, under the guidance of the trained teacher network model, the student network model is trained until the student network model converges to obtain a trained student network model;

[0016] Step 5), using the trained student network model as a new teacher network model, and selecting the next untrained, more complex neural network model from the set of neural network models to be trained as a new student network model, and then re-performing steps 2) to 4) until all neural network models in the set of neural network models to be trained have been trained;

[0017] Step 6) Based on the last trained student network model in the set of neural network models to be trained, the EEG signal classification model is obtained.

[0018] According to an EEG signal classification method provided by the present invention, the unlabeled EEG signal training data is input into the trained teacher network model, and according to the output result of the trained teacher network model, the unlabeled EEG signal training data is divided into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data, including:

[0019] Inputting the unlabeled EEG signal training data into the trained teacher network model to obtain a label prediction result; wherein the label prediction result is used to describe the probability value of the unlabeled EEG signal training data being determined to belong to the category;

[0020] Determining whether the label prediction result of the unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold;

[0021] In a case where the label prediction result of the unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, determining the unlabeled EEG signal training data as the first type of unlabeled EEG signal training data;

[0022] When the label prediction result of the unlabeled EEG signal training data is less than a predetermined confidence threshold, the unlabeled EEG signal training data is determined as the second type of unlabeled EEG signal training data.

[0023] According to an EEG signal classification method provided by the present invention, the unlabeled EEG signal training data is input into the trained teacher network model, and according to the output result of the teacher network model, the unlabeled EEG signal training data is divided into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data, including:

[0024] Inputting labeled EEG signal training data into the trained teacher network model to obtain a first mapping layer output result of the teacher network model;

[0025] Using a clustering algorithm to cluster the output results of the first mapping layer corresponding to all labeled EEG signal training data to obtain multiple categories, and calculating the cluster centers of the multiple categories;

[0026] Inputting the unlabeled EEG signal training data into the trained teacher network model to obtain a second mapping layer output result of the teacher network model;

[0027] Calculating the Euclidean distance between the second mapping layer output corresponding to the unlabeled EEG signal training data and the cluster center, and determining whether the Euclidean distance is greater than or equal to a predetermined confidence threshold,

[0028] When the Euclidean distance is less than or equal to a predetermined confidence threshold, determining the unlabeled EEG signal training data as the first type of unlabeled EEG signal training data;

[0029] When the Euclidean distance is greater than a predetermined confidence threshold, the unlabeled EEG signal training data is determined as second-category unlabeled EEG signal training data.

[0030] According to an EEG signal classification method provided by the present invention, the student network model is trained based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data and the second type of unlabeled EEG signal training data until the student network model converges to obtain a trained student network model, including:

[0031] Based on the student network model, comparative learning is performed on the second type of unlabeled EEG signal training data to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, and the KL divergence of the first output result and the second output result is calculated to obtain a first loss function;

[0032] Input the first type of unlabeled EEG signal training data into the trained teacher network model and the student network model respectively, and obtain the teacher network model output result and the student network model output result of the first type of unlabeled EEG signal training data; calculate the KL divergence of the teacher network model output result and the student network model output result to obtain a second loss function;

[0033] Inputting the labeled EEG signal training data into the student network model to obtain the output result of the student network model of the labeled EEG signal training data, calculating the cross entropy between the output result of the student network model of the labeled EEG signal training data and the label corresponding to the labeled EEG signal training data, and obtaining a third loss function;

[0034] Determine the total loss function of the student network model according to the first loss function, the second loss function and the third loss function;

[0035] The student network model is trained based on the total loss function until the student network model converges to obtain a trained student network model.

[0036] According to an EEG signal classification method provided by the present invention, the second type of unlabeled EEG signal training data is subjected to comparative learning based on the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, including:

[0037] Performing two data augmentation operations on the second type of unlabeled EEG signal training data to obtain a first data augmentation result and a second data augmentation result;

[0038] The first data augmentation result and the second data augmentation result are respectively input into the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data.

[0039] According to an EEG signal classification method provided by the present invention, the second type of unlabeled EEG signal training data is subjected to comparative learning based on the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, including:

[0040] Inputting the second type of unlabeled EEG signal training data into the student network model to obtain an output result of the student network model;

[0041] The output results of the student network model are randomly discarded twice to obtain the first output result and the second output result of the second type of unlabeled EEG signal training data.

[0042] According to an EEG signal classification method provided by the present invention, the EEG signal classification model is obtained based on the last trained student network model in the set of neural network models to be trained, including:

[0043] Selecting a feature extraction network from the last trained student network model in the set of neural network models to be trained;

[0044] A linear layer is added at the end of the feature extraction network to obtain the EEG signal classification model.

[0045] According to an EEG signal classification method provided by the present invention, before training the teacher network model with labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model, the method further includes:

[0046] Obtain original labeled EEG signal training data;

[0047] Preprocessing the labeled EEG signal training data, wherein the preprocessing includes at least one of the following operations: data desensitization, data denoising, data normalization, and data filtering;

[0048] Data augmentation is performed on the preprocessed labeled EEG signal training data to obtain data augmented labeled EEG signal training data.

[0049] The present invention also provides an EEG signal classification device, comprising: a signal acquisition module, used to acquire a target EEG signal to be processed;

[0050] A signal classification module, used for inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal;

[0051] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0052] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned methods for classifying electroencephalogram signals are implemented.

[0053] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any of the above-mentioned methods for classifying electroencephalogram signals are implemented.

[0054] The present invention also provides a computer program product, comprising a computer program, which implements the steps of any of the above-mentioned EEG signal classification methods when executed by a processor.

[0055] The EEG signal classification method, device, electronic device, medium and product provided by the present invention perform classification by using an EEG signal classification model. The EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times for training from multiple neural network models to be trained sorted by complexity, and the training is based on the student network model finally obtained by training. Therefore, under the conditions of a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, the teacher-student network method is used to train multiple neural network models to be trained, and finally an EEG signal classification model with higher classification accuracy is obtained, thereby obtaining a more accurate classification result, and can also greatly reduce the dependence of the deep learning model on high-quality data and reduce the data annotation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 is a flow chart of an EEG signal classification method provided by an embodiment of the present invention;

[0058] Figure 2 is a flowchart of the EEG signal classification model training process provided by an embodiment of the present invention;

[0059] Figure 3 is a schematic diagram of the prediction process of the trained teacher network model provided by an embodiment of the present invention;

[0060] Figure 4 is a schematic diagram of screening unlabeled EEG signal training data according to label prediction results provided by an embodiment of the present invention;

[0061] Figure 5 is a schematic diagram of screening unlabeled EEG signal training data by clustering provided by an embodiment of the present invention;

[0062] Figure 6 is a schematic diagram of a student network model training process provided by an embodiment of the present invention;

[0063] Figure 7A schematic diagram of the structure of an electroencephalogram signal classification device provided in an embodiment of the present invention;

[0064] Figure 8 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0066] Figure 1 is a flow chart of the EEG signal classification method provided by an embodiment of the present invention; Figure 1 As shown, the EEG signal classification method may include the following steps:

[0067] Step 101, obtaining a target EEG signal to be processed.

[0068] Specifically, the electrodes of the brain-computer interface can be used to directly obtain the EEG signals of the target object's head as the target EEG signals to be processed, or the target EEG signals to be processed can be obtained from a data storage device for storing EEG signals, which is not limited in this embodiment.

[0069] Step 102: input the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal.

[0070] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0071] Specifically, multiple (at least 2) neural network models to be trained are pre-constructed, and the neural network models to be trained are arranged in order of complexity from low to high according to the complexity of the models, and then a neural network model to be trained with low complexity is selected as a teacher network model, and a neural network model to be trained with high complexity is selected as a student network model in turn, and the teacher network model and the student network model are trained based on a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, and after the teacher network model and the student network model selected for the last time complete the training, the converged student network model is used as the EEG signal classification model.

[0072] In order to better illustrate the EEG signal classification model training process, this embodiment takes the pre-construction of three neural network models to be trained as an example for explanation:

[0073] First, three neural network models to be trained with different model complexities (different parameters) are constructed, and these three neural network models to be trained are arranged in ascending order of complexity to obtain neural network model 1 to be trained with the lowest complexity (fewest parameters), neural network model 2 to be trained with relatively general complexity, and neural network model 3 to be trained with the highest complexity (most parameters).

[0074] In the first round of training, the neural network model 1 to be trained is selected as the teacher network model, and the neural network model 2 to be trained is selected as the student network model. A small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data are used to train the teacher network model and the student network model to obtain a converged teacher network model and a converged student network model. At this time, the converged student network model is used as the new teacher network model, and the neural network model 3 to be trained is used as the new student network model for the second round of training.

[0075] In the second round of training, a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data are still used to train the new teacher network model and the new student network model, thereby obtaining a new converged teacher network model and a new converged student network model. Since all the neural network models to be trained have converged at this time, the converged student network model obtained in the second round of training is the student network model obtained by the final training. The student network model is used as the EEG signal classification model, and the target EEG signal to be processed is input into the EEG signal classification model, thereby obtaining the EEG signal classification result.

[0076] It should be noted that, in this embodiment, the EEG signal classification results are divided into 5 categories, namely, wakefulness (Wake, referred to as W), three non-rapid eye movement (Non-Rapid Eye Movement, referred to as N1, N2, N3) and one rapid eye movement (Rapid Eye Movement, referred to as REM). And in the EEG signal classification results output by the EEG signal classification model, the corresponding categories are represented by numbers 1-5. For example, if the EEG signal classification result output is 1, it means that the target EEG signal to be processed currently belongs to W (i.e., the EEG signal of the wakefulness period); by analogy, if the EEG signal classification result output is 2, it means that the target EEG signal to be processed currently belongs to N1 (i.e., the first non-rapid eye movement period). The above EEG signal classification results are classified according to the AASM standard. In other embodiments of the present invention, the EEG signal classification results can also be divided into wakefulness (Wake, W) according to the R&K standard, and sleep can be divided into rapid eye movement sleep phase (Rapid Eye Movement, REM) and non-rapid eye movement sleep phase (Non-Rapid Eye Movement, NREM). The non-rapid eye movement period can be divided into four periods: sleep stage 1 (S1) to sleep stage 4 (S4). S1 and S2 are light sleep, and S3 and S4 are deep sleep.

[0077] The EEG signal classification method provided by the embodiment of the present invention performs classification by using an EEG signal classification model. The EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times for training from multiple neural network models to be trained sorted by complexity, and the training is based on the student network model finally obtained by training. Therefore, under the conditions of a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, the teacher-student network method is used to train multiple neural network models to be trained, and finally an EEG signal classification model with higher classification accuracy is obtained, thereby obtaining a more accurate classification result, and greatly reducing the dependence of the deep learning model on high-quality data and reducing the data annotation cost.

[0078] Figure 2 is a flow chart of the EEG signal classification model training process provided by an embodiment of the present invention; Figure 2 As shown, before the target EEG signal to be processed is input into the EEG signal classification model, the method further includes:

[0079] Step 1), selecting a teacher network model and a student network model from a set of neural network models to be trained; wherein the complexity of the teacher network model is lower than the complexity of the student network model; the neural network models in the set of neural network models to be trained are sorted in order of complexity;

[0080] Step 2), training the teacher network model using labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model;

[0081] Step 3), inputting the unlabeled EEG signal training data into the trained teacher network model, and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to the output result of the trained teacher network model; wherein the confidence of the first category of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second category of unlabeled EEG signal training data is less than the predetermined confidence threshold;

[0082] Step 4), based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data and the second type of unlabeled EEG signal training data, under the guidance of the trained teacher network model, the student network model is trained until the student network model converges to obtain a trained student network model;

[0083] Step 5), using the trained student network model as a new teacher network model, and selecting the next untrained, more complex neural network model from the set of neural network models to be trained as a new student network model, and then re-performing steps 2) to 4) until all neural network models in the set of neural network models to be trained have been trained;

[0084] Step 6) Based on the last trained student network model in the set of neural network models to be trained, the EEG signal classification model is obtained.

[0085] Specifically, step 1): first construct three to ten neural network models to be trained to form a set of neural network models to be trained, and arrange the models in the set from simple to complex according to complexity. Then, select the first (i.e., the one with the lowest complexity) neural network model to be trained as the teacher network model, and the second (i.e., the second lowest complexity) neural network model to be trained as the student network model, as the training object in the first round of training, and the remaining neural network models to be trained in the set are used as new student network models in subsequent multiple rounds of training.

[0086] Among them, each neural network model to be trained includes at least a feature extraction layer, a mapping layer and a classification layer. And the feature extraction layer adopts a deep learning convolutional neural network method, which extracts high-dimensional abstract information of the data through a convolution layer, a pooling layer, an activation layer, and a batch normalization layer, so that after the data is input into the network, the deep semantic information of the data can be extracted. The second layer of each neural network model to be trained is a mapping layer. A fully connected neural network mapping layer is designed after the aforementioned first layer of feature extraction layer, which can extract the deep semantic information of the EEG signal again. The classification layer is generally a fully connected layer, and a softmax activation function is connected after the classification layer to classify the high-level features of the EEG signal.

[0087] Step 2): Use a small amount of labeled EEG signal training data to train the teacher network model until convergence to obtain a trained teacher network model. During the training process, ensure that the teacher network model does not appear overfitting or underfitting.

[0088] Specifically, a small amount of labeled EEG signal training data is input into the teacher network model, and the cross entropy is used as the loss function to optimize the teacher network model. The loss function is shown in formula (1):

[0089]

[0090] In the formula, is the data calculated after the labeled EEG signal training data passes through the classification layer in the teacher network model, y label is the label corresponding to the labeled EEG signal training data, y pred Yes After doing SoftMax, take the column label of the maximum probability, N is the number of samples, and C is the number of classification categories.

[0091] According to the loss function shown in formula (1) and the labels corresponding to the labeled EEG signal training data, the teacher network model is trained using the back propagation algorithm until the teacher network model converges to obtain a trained teacher network model.

[0092] Step 3): First, use the trained teacher network model to predict the unlabeled EEG signal training data to obtain the label prediction result (i.e., the pseudo label of the unlabeled EEG signal training data). Then, filter the unlabeled EEG signal training data according to the pseudo label to obtain the first category of unlabeled EEG signal training data and the second category of unlabeled EEG signal training data.

[0093] The confidence of the first type of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second type of unlabeled EEG signal training data is less than the predetermined confidence threshold.

[0094] Figure 3 is a schematic diagram of the prediction process of the trained teacher network model provided by an embodiment of the present invention; Figure 3 As shown in the figure, the unlabeled EEG signal training data is predicted using the trained teacher network model to obtain the label prediction result. θt , mapping layer g θt And the classification layer h θt The unlabeled EEG signal training data x is calculated, and the calculation results obtained at each layer are y θt 、z θt ,q θt (z θt ).

[0095] The specific screening method can be to select the q calculated by the classification layer in the trained teacher network model θt (z θt ) is greater than or equal to a certain threshold as the first type of unlabeled EEG signal training data (i.e., high confidence pseudo-label data), and the unlabeled EEG signal training data less than the threshold is used as the second type of unlabeled EEG signal training data (i.e., other unlabeled data); cluster selection can also be performed based on the characteristic distribution of pseudo-labels, and clusters with a distance from the cluster centroid less than or equal to a certain threshold (here, the same as "q θt (z θt ) is greater than or equal to a certain threshold. The thresholds in the equation are not the same. The unlabeled EEG signal training data corresponding to the pseudo-label of ) is used as the first category of unlabeled EEG signal training data (i.e., high confidence pseudo-label data), and the unlabeled EEG signal training data greater than the threshold is used as the second category of unlabeled EEG signal training data (i.e., other unlabeled data).

[0096] Step 4): Use a small amount of labeled data and the first type of unlabeled EEG signal training data (i.e., high-confidence pseudo-label data) and train the student network model under the guidance of the trained teacher network model. Use the second type of unlabeled EEG signal training data to adjust the parameters in the student network model until the student network model converges to obtain a trained student network model.

[0097] Step 5): Use the trained student network model as the new teacher network model, and select the next untrained and more complex neural network model from the set of neural network models to be trained as the new student network model according to the complexity of the model, and then re-execute steps 2)-4) to start the next round of training until all neural network models in the set of neural network models to be trained have been trained.

[0098] Step 6): The last trained student network model in step 5) is used as the EEG signal classification model.

[0099] After completing the training of the EEG signal classification model, the EEG signal classification model is tested and verified using the EEG signal test set to obtain the classification results to calculate indicators such as accuracy, precision, and recall rate, or the model is used for feature extraction and feature analysis to evaluate the performance of the EEG signal classification model.

[0100] The EEG signal classification method provided in the embodiment of the present invention trains multiple neural network models to be trained with different model complexities in a teacher-student network manner under the conditions of a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, and uses the last trained student network model as the EEG signal classification model, thereby reducing the dependence on high-quality data, obtaining a classification model with better performance, and reducing the data labeling cost.

[0101] Furthermore, the step of inputting the unlabeled EEG signal training data into the trained teacher network model, and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to an output result of the trained teacher network model, comprises:

[0102] Inputting the unlabeled EEG signal training data into the trained teacher network model to obtain a label prediction result; wherein the label prediction result is used to describe the probability value of the unlabeled EEG signal training data being determined to belong to the category;

[0103] Determining whether the label prediction result of the unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold;

[0104] In a case where the label prediction result of the unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, determining the unlabeled EEG signal training data as the first type of unlabeled EEG signal training data;

[0105] When the label prediction result of the unlabeled EEG signal training data is less than a predetermined confidence threshold, the unlabeled EEG signal training data is determined as the second type of unlabeled EEG signal training data.

[0106] Figure 4 is a schematic diagram of screening unlabeled EEG signal training data according to label prediction results provided by an embodiment of the present invention; Figure 4 As shown in the figure, a confidence threshold (e.g. 0.7) is set in advance to compare the q calculated by the classification layer in the trained teacher network model with unlabeled data. θt (zθt ) value (i.e., the label prediction result, which is used to describe the probability value of the unlabeled EEG signal training data being determined as the category to which it belongs, and generally the maximum value is taken as the prediction result) and the confidence threshold. If q θt (z θt ) as a high confidence pseudo-label dataset, denoted as X unlabel1 (i.e. the first type of unlabeled EEG signal training data), and the data sets other than the unlabeled data set are recorded as X unlabel2 (i.e. the second type of unlabeled EEG signal training data).

[0107] It should be noted that, in other embodiments of the present invention, the confidence threshold may also be an interval. If the value is within the confidence threshold [a1, a2], then X unlabel1 , if it is less than a1 or greater than a2, it is X unlabel2 .

[0108] In addition, in this embodiment A value greater than or equal to the confidence threshold is high confidence, and a value less than the confidence threshold is low confidence, so as to filter out unlabeled data. In other embodiments of the present invention, other methods for distinguishing high confidence from low confidence can also be used to filter out the first category of unlabeled EEG signal training data with high confidence and the second category of unlabeled EEG signal training data with low confidence.

[0109] The EEG signal classification method provided in the embodiment of the present invention classifies and predicts the unlabeled EEG signal training data through a trained teacher network model to obtain a label prediction result, and screens the unlabeled EEG signal training data according to the maximum probability value of the category to which it belongs and a preset confidence threshold. The first category of unlabeled EEG signal training data obtained and its corresponding label prediction result are of better quality. During the training process, the student network model will not be misled, and the performance of the student network model can be improved.

[0110] Further, the unlabeled EEG signal training data is input into the trained teacher network model, and according to the output result of the teacher network model, the unlabeled EEG signal training data is divided into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data, including:

[0111] Inputting labeled EEG signal training data into the trained teacher network model to obtain a first mapping layer output result of the teacher network model;

[0112] Using a clustering algorithm to cluster the output results of the first mapping layer corresponding to all labeled EEG signal training data to obtain multiple categories, and calculating the cluster centers of the multiple categories;

[0113] Inputting the unlabeled EEG signal training data into the trained teacher network model to obtain a second mapping layer output result of the teacher network model;

[0114] Calculating the Euclidean distance between the second mapping layer output corresponding to the unlabeled EEG signal training data and the cluster center, and determining whether the Euclidean distance is greater than or equal to a predetermined confidence threshold,

[0115] When the Euclidean distance is less than or equal to a predetermined confidence threshold, determining the unlabeled EEG signal training data as the first type of unlabeled EEG signal training data;

[0116] When the Euclidean distance is greater than a predetermined confidence threshold, the unlabeled EEG signal training data is determined as second-category unlabeled EEG signal training data.

[0117] Figure 5 is a schematic diagram of screening unlabeled EEG signal training data by clustering provided by an embodiment of the present invention; Figure 5 As shown, there will be labeled data x label (i.e. labeled EEG signal training data) input the trained teacher network model to obtain the calculation results of the mapping layer output of the teacher network model (i.e. the output result of the first mapping layer), K-means is used for clustering to obtain five categories (corresponding to the five categories W, N1, N2, N3 and REM mentioned above), and the cluster centers and average class distances of the five categories are calculated. The classification performance of the model can be evaluated by the average inter-class distance and the average intra-class distance.

[0118] Then the unlabeled data X unlabel The feature vector output by the mapping layer obtained by inputting the trained teacher network (i.e., unlabeled EEG signal training data) (i.e., the output result of the second mapping layer), the feature vector Then, calculate z θt2 Euclidean distance d to the five cluster centers ij And determine a confidence threshold d i0 , select the five cluster centers that are closer to the threshold (i.e., the Euclidean distance d ij Less than or equal to the predetermined confidence threshold d i0 ) forms a dataset X unlabel1 (i.e. the first type of unlabeled EEG signal training data), and the data sets other than the unlabeled data set are recorded as X unlabel2 (i.e. the second type of unlabeled EEG signal training data).

[0119] In this embodiment, the confidence threshold d i0 is the average distance from the data points to the centroid in the data cluster.

[0120] It should be noted that in this embodiment, the Euclidean distance d ij Less than or equal to the confidence threshold d i0 That is, the confidence level of the data is high, which is greater than the confidence threshold d i0 That is, the confidence of the data is low, so the unlabeled data is screened. In other embodiments of the present invention, other high-confidence and low-confidence distinction methods can also be used to screen out the first category of unlabeled EEG signal training data with high confidence and the second category of unlabeled EEG signal training data with low confidence.

[0121] In addition, the number of categories of the cluster center corresponds to the number of categories of the EEG signal classification results. If the categories of the EEG signal classification results are divided according to the R&K standard, the number of categories of the corresponding cluster center is also 6.

[0122] The EEG signal classification method provided by the embodiment of the present invention clusters the results output by the mapping layer of the trained teacher network model of the labeled EEG signal training data, and analyzes the unlabeled EEG signal training data according to the clustering results, thereby screening out a first category of unlabeled EEG signal training data. The category to which the first category of unlabeled EEG signal training data belongs is closer to the category described by the labeled EEG signal training data, thereby improving the quality of the first category of unlabeled EEG signal training data. During the training process, the student network model will not be misled, and the performance of the student network model can be improved.

[0123] Furthermore, the student network model is trained based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data, and the second type of unlabeled EEG signal training data until the student network model converges to obtain a trained student network model, including:

[0124] Based on the student network model, comparative learning is performed on the second type of unlabeled EEG signal training data to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, and the KL divergence of the first output result and the second output result is calculated to obtain a first loss function;

[0125] Input the first type of unlabeled EEG signal training data into the trained teacher network model and the student network model respectively, and obtain the teacher network model output result and the student network model output result of the first type of unlabeled EEG signal training data; calculate the KL divergence of the teacher network model output result and the student network model output result to obtain a second loss function;

[0126] Inputting the labeled EEG signal training data into the student network model to obtain the output result of the student network model of the labeled EEG signal training data, calculating the cross entropy between the output result of the student network model of the labeled EEG signal training data and the label corresponding to the labeled EEG signal training data, and obtaining a third loss function;

[0127] Determine the total loss function of the student network model according to the first loss function, the second loss function and the third loss function;

[0128] The student network model is trained based on the total loss function until the student network model converges to obtain a trained student network model.

[0129] Figure 6 It is a schematic diagram of the student network model training process provided by an embodiment of the present invention.

[0130] like Figure 6 As shown in the figure, first, the student network model is used to train the second type of unlabeled EEG signal data X unlabel2 Perform comparative learning to obtain the first output results output by the classification layer of the student network model With the second output result calculate and The KL divergence between them gives the first loss function loss1:

[0131]

[0132] In the formula, for The probability values ​​of the corresponding dimensions of the two vectors after taking SoftMax, N is the total number of samples, and C is the total number of categories.

[0133] Then, the first type of unlabeled EEG signal training data X unlabel1 Input the trained teacher network model respectively and student network model go through The classification layer and After the classification layer, the output results of the teacher network model are obtained respectively. Output results of the student network model calculate and The KL divergence between them gives the second loss function loss2:

[0134]

[0135] In the formula, for The probability values ​​of the corresponding dimensions of the two vectors after taking SoftMax, N is the total number of samples, and C is the total number of categories.

[0136] Next, the labeled EEG signal training data X label Input the student network to get the classification layer output: (i.e., the output of the student network model of labeled EEG signal training data), calculate With X label The corresponding label y label The cross entropy loss function loss3 (the third loss function) between:

[0137]

[0138] In the formula, y pred Yes After doing SoftMax, take the column label of the maximum probability, N is the number of samples, and C is the number of classification categories.

[0139] Then, the total loss function L is determined based on the first loss function, the second loss function, and the third loss function. loss :

[0140] L loss =a*loss1+b*loss2+c*loss3 (5)

[0141] Where a, b, and c are corresponding constants, representing the weighted average coefficients.

[0142] Finally, according to the total loss function L loss The student network model is trained until the student network model converges to obtain a trained student network model.

[0143] It should be noted that, in the above-mentioned student network model training process, the parameter update is as shown in formula (6), where optimizer is the optimizer, θ is the neural network parameter, and η is the learning rate.

[0144]

[0145] In addition, set all model parameters to be gradient variable, and set appropriate hyperparameters such as optimizer, learning rate, number of training segments, and training rounds.

[0146] The accuracy, recall, precision and other indicators of all models are calculated on the test data set until the model converges and the accuracy of the test set is stable, thus obtaining the calculation accuracy, precision, recall and other indicators of the student network model. The trained student network model can also be used for feature extraction and feature analysis to evaluate model performance.

[0147] The EEG signal classification method provided in the embodiment of the present invention improves the accuracy of the student network model by contrasting the learning method and three loss functions.

[0148] Furthermore, the second type of unlabeled EEG signal training data is subjected to comparative learning based on the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, including:

[0149] Performing two data augmentation operations on the second type of unlabeled EEG signal training data to obtain a first data augmentation result and a second data augmentation result;

[0150] The first data augmentation result and the second data augmentation result are respectively input into the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data.

[0151] Specifically, for the second type of unlabeled EEG signal training data X unlabel2 After two data enhancements, the first data augmentation result data X is obtained aug1 and the second data augmentation result X aug2 . X aug1 With X aug2 Input them into the student network model respectively, and get the output of two classification layers accordingly: the first output result With the second output result

[0152] The EEG signal classification method provided in the embodiment of the present invention realizes comparative learning by means of data augmentation, thereby improving the accuracy of the model.

[0153] Furthermore, the second type of unlabeled EEG signal training data is subjected to comparative learning based on the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, including:

[0154] Inputting the second type of unlabeled EEG signal training data into the student network model to obtain an output result of the student network model;

[0155] The output results of the student network model are randomly discarded twice to obtain the first output result and the second output result of the second type of unlabeled EEG signal training data.

[0156] Specifically, the second type of unlabeled EEG signal training data X unlabel2 Input the student network model to get the output result of the student network model, perform two random dropouts (i.e., randomly discard the features) to get the first output result With the second output result

[0157] It should be noted that the Dropout in this embodiment is to randomly retain a portion of information as information for subsequent processing according to a certain ratio after inputting the data into the model to obtain data features. This operation is performed once to obtain two sets of data features. Since they are from the same data, the retained data features here will have similarities, which facilitates the subsequent comparative learning loss calculation. Compared with the traditional method of comparison through data enhancement, only one model is involved in the calculation and parameter adjustment, so the calculation complexity will be much smaller, but the model effect will not change much.

[0158] The EEG signal classification method provided in an embodiment of the present invention discards features of the output results of the student network model through the Dropout method, thereby obtaining a feature vector after dimensionality reduction, and realizes comparative learning based on the two feature vectors after dimensionality reduction, thereby further improving the accuracy of the student network model.

[0159] Furthermore, the electroencephalogram signal classification model is obtained based on the last trained student network model in the set of neural network models to be trained, including:

[0160] Selecting a feature extraction network from the last trained student network model in the set of neural network models to be trained;

[0161] A linear layer is added at the end of the feature extraction network to obtain the EEG signal classification model.

[0162] Specifically, a feature extraction network is selected from the last trained student network model, and a linear layer is added to the end of the feature extraction network to obtain an EEG signal classification model.

[0163] Furthermore, before training the teacher network model using labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model, the method further includes:

[0164] Obtain original labeled EEG signal training data;

[0165] Preprocessing the labeled EEG signal training data, wherein the preprocessing includes at least one of the following operations: data desensitization, data denoising, data normalization, and data filtering;

[0166] Data augmentation is performed on the preprocessed labeled EEG signal training data to obtain data augmented labeled EEG signal training data.

[0167] Specifically, the original labeled EEG signal training data is first obtained, and then the integrity and availability of the labeled EEG signal training data are checked, and the labeled EEG signal training data is preprocessed through methods such as data denoising, desensitization, data access, data repair, bandpass filter filtering, LMS adaptive filtering, Wiener filtering, filtering method based on statistical model, filtering method based on neural network, maximum and minimum normalization, dimensionality reduction, wavelet transform, etc.

[0168] Next, a small amount of labeled EEG signal training data is augmented by performing data augmentation such as dimension change, time shift, Gaussian noise addition, pulse waveform addition, cropping, speed change, etc. to obtain the labeled EEG signal training data after data augmentation.

[0169] The integrity check of labeled EEG signal training data can be judged according to the corresponding data integrity detection rules, or according to the distribution regularity of the data, the data that does not conform to the distribution regularity or is disordered is regarded as incomplete data, and the rest is regarded as complete data. The incomplete data obtained by the two inspection methods are eliminated to ensure data quality.

[0170] The specific preprocessing process is as follows: if the sensitive data involves information security or user privacy, data desensitization needs to be performed; if the data is signal data, bandpass filtering, wavelet transform, signal framing and other processing methods are used; if the data is image data, dimensionality change and other processing are performed.

[0171] The EEG signal classification method provided by the embodiment of the present invention improves the quality of data through preprocessing and increases the generalization of data through data augmentation, thereby improving the generalization, robustness and accuracy of the model.

[0172] Figure 7 A schematic diagram of the structure of an EEG signal classification device provided in an embodiment of the present invention is shown in FIG. Figure 7 As shown, an electroencephalogram signal classification device comprises:

[0173] The signal acquisition module 701 is used to acquire the target EEG signal to be processed.

[0174] Specifically, the signal acquisition module 701 can use the electrodes of the brain-computer interface to directly acquire the EEG signal of the target object's head as the target EEG signal to be processed, or it can acquire the target EEG signal to be processed from a data storage device used to store EEG signals. This embodiment is not limited to this.

[0175] The signal classification module 702 is used to input the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal.

[0176] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0177] Specifically, multiple (at least 2) neural network models to be trained are pre-constructed, and the neural network models to be trained are arranged in order of complexity from low to high according to the complexity of the models, and then a neural network model to be trained with low complexity is selected as a teacher network model, and a neural network model to be trained with high complexity is selected as a student network model in turn, and the teacher network model and the student network model are trained based on a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, and after the teacher network model and the student network model selected for the last time complete the training, the converged student network model is used as the EEG signal classification model.

[0178] The EEG signal classification device provided by the embodiment of the present invention performs classification by using an EEG signal classification model. The EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times for training from multiple neural network models to be trained sorted by complexity, and the training is based on the student network model finally obtained by training. Therefore, under the conditions of a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, the teacher-student network method is used to train multiple neural network models to be trained, and finally an EEG signal classification model with higher classification accuracy is obtained, thereby obtaining a more accurate classification result, and greatly reducing the dependence of the deep learning model on high-quality data and reducing the data annotation cost.

[0179] Furthermore, the device also includes:

[0180] A teacher and student network selection unit, used to select a teacher network model and a student network model from a set of neural network models to be trained; wherein the complexity of the teacher network model is lower than the complexity of the student network model; and the neural network models in the set of neural network models to be trained are sorted in order of complexity;

[0181] A teacher network model training unit, used to train the teacher network model through labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model;

[0182] An unlabeled data classification unit, used for inputting unlabeled EEG signal training data into the trained teacher network model, and classifying the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to an output result of the trained teacher network model; wherein the confidence of the first category of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second category of unlabeled EEG signal training data is less than the predetermined confidence threshold;

[0183] A student network model training unit, configured to train the student network model based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data, and the second type of unlabeled EEG signal training data, under the guidance of the trained teacher network model, until the student network model converges to obtain a trained student network model;

[0184] A model retraining unit, used to use the trained student network model as a new teacher network model, and select the next untrained and more complex neural network model from the set of neural network models to be trained as a new student network model, and then re-execute the teacher network model training unit-student network model training unit until all neural network models in the set of neural network models to be trained have been trained;

[0185] The EEG signal classification model determination unit is used to obtain the EEG signal classification model based on the last trained student network model in the set of neural network models to be trained.

[0186] Specifically, the teacher and student network selection unit: first construct three to ten neural network models to be trained to form a set of neural network models to be trained, and arrange the models in the set from simple to complex in terms of complexity. Then, select the first (i.e., the one with the lowest complexity) neural network model to be trained as the teacher network model, and the second (i.e., the second lowest complexity) neural network model to be trained as the student network model, as the training object in the first round of training, and the remaining neural network models to be trained in the set as new student network models in subsequent rounds of training.

[0187] Among them, each neural network model to be trained includes at least a feature extraction layer, a mapping layer and a classification layer. And the feature extraction layer adopts a deep learning convolutional neural network method, which extracts high-dimensional abstract information of the data through a convolution layer, a pooling layer, an activation layer, and a batch normalization layer, so that after the data is input into the network, the deep semantic information of the data can be extracted. The second layer of each neural network model to be trained is a mapping layer. A fully connected neural network mapping layer is designed after the aforementioned first layer of feature extraction layer, which can extract the deep semantic information of the EEG signal again. The classification layer is generally a fully connected layer, and a softmax activation function is connected after the classification layer to classify the high-level features of the EEG signal.

[0188] Teacher network model training unit: Use a small amount of labeled EEG signal training data to train the teacher network model until convergence to obtain a trained teacher network model. During the training process, ensure that the teacher network model does not appear overfitting or underfitting.

[0189] Specifically, a small amount of labeled EEG signal training data is input into the teacher network model, and the cross entropy is used as the loss function to optimize the teacher network model. The loss function is shown in formula (1):

[0190]

[0191] In the formula, is the data calculated after the labeled EEG signal training data passes through the classification layer in the teacher network model, y pred Yes After doing SoftMax, take the column label of the maximum probability, N is the number of samples, and C is the number of classification categories.

[0192] According to the loss function shown in formula (1) and the labels corresponding to the labeled EEG signal training data, the teacher network model is trained using the back propagation algorithm until the teacher network model converges to obtain a trained teacher network model.

[0193] Unlabeled data classification unit: First, use the trained teacher network model to predict the unlabeled EEG signal training data to obtain the label prediction result (i.e., the pseudo label of the unlabeled EEG signal training data). Then, the unlabeled EEG signal training data is screened according to the pseudo label to obtain the first category of unlabeled EEG signal training data and the second category of unlabeled EEG signal training data.

[0194] The confidence of the first type of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second type of unlabeled EEG signal training data is less than the predetermined confidence threshold.

[0195] Specifically, the trained teacher network model is used to predict the unlabeled EEG signal training data to obtain the label prediction result. θt , mapping layer g θt And the classification layer h θt The unlabeled EEG signal training data x is calculated, and the calculation results obtained at each layer are y θt 、z θt ,q θt (z θt ).

[0196] The specific screening method can be to select The unlabeled EEG signal training data greater than or equal to a certain threshold is used as the first category of unlabeled EEG signal training data (i.e., high-confidence pseudo-label data), and the unlabeled EEG signal training data less than the threshold is used as the second category of unlabeled EEG signal training data (i.e., other unlabeled data); cluster selection can also be performed based on the characteristic distribution of the pseudo-label, and the unlabeled EEG signal training data corresponding to the pseudo-label whose distance from the cluster centroid is less than or equal to a certain threshold (the two thresholds are different) is selected as the first category of unlabeled EEG signal training data (i.e., high-confidence pseudo-label data), and the unlabeled EEG signal training data greater than the threshold is used as the second category of unlabeled EEG signal training data (i.e., other unlabeled data).

[0197] Student network model training unit: using a small amount of labeled data and the first type of unlabeled EEG signal training data (i.e., high-confidence pseudo-label data), and under the guidance of the trained teacher network model, the student network model is trained, and the parameters in the student network model are adjusted using the second type of unlabeled EEG signal training data until the student network model converges to obtain a trained student network model.

[0198] Model repeated training unit: the trained student network model is used as the new teacher network model, and the next untrained and more complex neural network model is selected from the set of neural network models to be trained according to the complexity of the model as the new student network model, and then the teacher network model training unit-student network model training unit is re-executed to start the next round of training until all neural network models in the set of neural network models to be trained have been trained.

[0199] The EEG signal classification model determination unit uses the last trained student network model in the model repetition training unit as the EEG signal classification model.

[0200] After completing the training of the EEG signal classification model, the EEG signal classification model is tested and verified using the EEG signal test set to obtain the classification results to calculate indicators such as accuracy, precision, and recall rate, or the model is used for feature extraction and feature analysis to evaluate the performance of the EEG signal classification model.

[0201] The EEG signal classification device provided by the embodiment of the present invention trains multiple neural network models to be trained with different model complexities in a teacher-student network manner under the conditions of a small amount of labeled EEG signal training data and a large amount of unlabeled EEG signal training data, and uses the last trained student network model as the EEG signal classification model, thereby reducing the dependence on high-quality data, obtaining a classification model with better performance, and reducing the data labeling cost.

[0202] Figure 8 A schematic diagram of the physical structure of an electronic device provided by an embodiment of the present invention, such as Figure 8 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820 and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call the logic instructions in the memory 830 to execute the EEG signal classification method, including: obtaining the target EEG signal to be processed;

[0203] Inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal;

[0204] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0205] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0206] On the other hand, the present invention also provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the EEG signal classification method provided by the above methods, including: obtaining a target EEG signal to be processed;

[0207] Inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal;

[0208] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0209] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for performing the EEG signal classification method provided by the above methods is implemented, including: obtaining a target EEG signal to be processed;

[0210] Inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal;

[0211] Among them, the EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data. A teacher network model and a student network model are selected multiple times from multiple neural network models to be trained sorted by complexity for training, and the model is obtained based on the student network model finally trained.

[0212] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0213] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0214] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for classifying electroencephalogram signals, characterized in that: include: Acquire the target EEG signal to be processed; Inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal; The EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data, and is obtained by repeatedly selecting a teacher network model and a student network model from a plurality of neural network models to be trained sorted by complexity, and is based on the student network model finally obtained by training; Before inputting the target EEG signal to be processed into the EEG signal classification model, the method further includes: Step 1), selecting a teacher network model and a student network model from a set of neural network models to be trained; wherein the complexity of the teacher network model is lower than the complexity of the student network model; the neural network models in the set of neural network models to be trained are sorted in order of complexity; Step 2), training the teacher network model using labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model; Step 3), inputting the unlabeled EEG signal training data into the trained teacher network model, and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to the output result of the trained teacher network model; wherein the confidence of the first category of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second category of unlabeled EEG signal training data is less than the predetermined confidence threshold; Step 4), based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data and the second type of unlabeled EEG signal training data, under the guidance of the trained teacher network model, the student network model is trained until the student network model converges to obtain a trained student network model; Step 5), using the trained student network model as a new teacher network model, and selecting the next untrained and more complex neural network model from the set of neural network models to be trained as a new student network model, and then re-performing steps 2) to 4), until all neural network models in the set of neural network models to be trained have been trained; Step 6), based on the last trained student network model in the set of neural network models to be trained, the EEG signal classification model is obtained.

2. The method for classifying EEG signals according to claim 1, characterized in that: The step of inputting the unlabeled EEG signal training data into the trained teacher network model and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to an output result of the trained teacher network model comprises: Inputting the unlabeled EEG signal training data into the trained teacher network model to obtain a label prediction result; wherein the label prediction result is used to describe the probability value of the unlabeled EEG signal training data being determined to belong to the category; Determining whether the label prediction result of the unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold; In a case where the label prediction result of the unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, determining the unlabeled EEG signal training data as the first type of unlabeled EEG signal training data; When the label prediction result of the unlabeled EEG signal training data is less than a predetermined confidence threshold, the unlabeled EEG signal training data is determined as the second type of unlabeled EEG signal training data.

3. The method for classifying EEG signals according to claim 1, characterized in that: The step of inputting the unlabeled EEG signal training data into the trained teacher network model and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to an output result of the teacher network model comprises: Inputting labeled EEG signal training data into the trained teacher network model to obtain a first mapping layer output result of the teacher network model; Using a clustering algorithm to cluster the output results of the first mapping layer corresponding to all labeled EEG signal training data to obtain multiple categories, and calculating the cluster centers of the multiple categories; Inputting the unlabeled EEG signal training data into the trained teacher network model to obtain a second mapping layer output result of the teacher network model; Calculating the Euclidean distance between the second mapping layer output corresponding to the unlabeled EEG signal training data and the cluster center, and determining whether the Euclidean distance is greater than or equal to a predetermined confidence threshold, When the Euclidean distance is less than or equal to a predetermined confidence threshold, determining the unlabeled EEG signal training data as the first type of unlabeled EEG signal training data; When the Euclidean distance is greater than a predetermined confidence threshold, the unlabeled EEG signal training data is determined as second-category unlabeled EEG signal training data.

4. The method for classifying EEG signals according to claim 1, characterized in that: The step of training the student network model based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data, and the second type of unlabeled EEG signal training data until the student network model converges to obtain a trained student network model includes: Based on the student network model, comparative learning is performed on the second type of unlabeled EEG signal training data to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data, and the KL divergence of the first output result and the second output result is calculated to obtain a first loss function; Input the first type of unlabeled EEG signal training data into the trained teacher network model and the student network model respectively, and obtain the teacher network model output result and the student network model output result of the first type of unlabeled EEG signal training data; calculate the KL divergence of the teacher network model output result and the student network model output result to obtain a second loss function; Inputting the labeled EEG signal training data into the student network model to obtain the output result of the student network model of the labeled EEG signal training data, calculating the cross entropy between the output result of the student network model of the labeled EEG signal training data and the label corresponding to the labeled EEG signal training data, and obtaining a third loss function; Determine the total loss function of the student network model according to the first loss function, the second loss function and the third loss function; The student network model is trained based on the total loss function until the student network model converges to obtain a trained student network model.

5. The method for classifying EEG signals according to claim 4, characterized in that: The method of performing comparative learning on the second type of unlabeled EEG signal training data based on the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data includes: Performing two data augmentation operations on the second type of unlabeled EEG signal training data to obtain a first data augmentation result and a second data augmentation result; The first data augmentation result and the second data augmentation result are respectively input into the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data.

6. The method for classifying EEG signals according to claim 4, characterized in that: The method of performing comparative learning on the second type of unlabeled EEG signal training data based on the student network model to obtain a first output result and a second output result of the second type of unlabeled EEG signal training data includes: Inputting the second type of unlabeled EEG signal training data into the student network model to obtain an output result of the student network model; The output results of the student network model are randomly discarded twice to obtain the first output result and the second output result of the second type of unlabeled EEG signal training data.

7. The method for classifying EEG signals according to claim 1, characterized in that: The method of obtaining the EEG signal classification model based on the last trained student network model in the set of neural network models to be trained comprises: Selecting a feature extraction network from the last trained student network model in the set of neural network models to be trained; A linear layer is added at the end of the feature extraction network to obtain the EEG signal classification model.

8. The method for classifying EEG signals according to claim 1, characterized in that: Before training the teacher network model with labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model, the method further includes: Obtain original labeled EEG signal training data; Preprocessing the labeled EEG signal training data, wherein the preprocessing includes at least one of the following operations: data desensitization, data denoising, data normalization, and data filtering; Data augmentation is performed on the preprocessed labeled EEG signal training data to obtain data augmented labeled EEG signal training data.

9. An electroencephalogram signal classification device, characterized in that: include: A signal acquisition module, used to acquire the target EEG signal to be processed; A signal classification module, used for inputting the target EEG signal to be processed into an EEG signal classification model to obtain an EEG signal classification result corresponding to the target EEG signal; The EEG signal classification model is based on labeled EEG signal training data and unlabeled EEG signal training data, and is obtained by repeatedly selecting a teacher network model and a student network model from a plurality of neural network models to be trained sorted by complexity, and is based on the student network model finally obtained by training; Before inputting the target EEG signal to be processed into the EEG signal classification model, the method further includes: Step 1), selecting a teacher network model and a student network model from a set of neural network models to be trained; wherein the complexity of the teacher network model is lower than the complexity of the student network model; the neural network models in the set of neural network models to be trained are sorted in order of complexity; Step 2), training the teacher network model using labeled EEG signal training data until the teacher network model converges to obtain a trained teacher network model; Step 3), inputting the unlabeled EEG signal training data into the trained teacher network model, and dividing the unlabeled EEG signal training data into a first category of unlabeled EEG signal training data and a second category of unlabeled EEG signal training data according to the output result of the trained teacher network model; wherein the confidence of the first category of unlabeled EEG signal training data is greater than or equal to a predetermined confidence threshold, and the confidence of the second category of unlabeled EEG signal training data is less than the predetermined confidence threshold; Step 4), based on the labeled EEG signal training data, the first type of unlabeled EEG signal training data and the second type of unlabeled EEG signal training data, under the guidance of the trained teacher network model, the student network model is trained until the student network model converges to obtain a trained student network model; Step 5), using the trained student network model as a new teacher network model, and selecting the next untrained and more complex neural network model from the set of neural network models to be trained as a new student network model, and then re-performing steps 2) to 4), until all neural network models in the set of neural network models to be trained have been trained; Step 6), based on the last trained student network model in the set of neural network models to be trained, the EEG signal classification model is obtained.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the EEG signal classification method according to any one of claims 1 to 8 are implemented.

11. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the EEG signal classification method according to any one of claims 1 to 8 are implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the EEG signal classification method according to any one of claims 1 to 8 are implemented.

Citation Information

Patent Citations

  • Pure idea control intelligent rehabilitation method based on distillation learning and deep learning and application

    CN111544855A

  • Semi-supervised classification method and device based on deep learning, and storage medium

    CN112699960A

  • Knowledge distillation using deep clustering

    GB202116914D0