Method and apparatus for training EEG classification model using a two-phase multi-task autoencoder

Abstract

Disclosed are a method and apparatus for training an EEG classification model by using a two-phase multi-task autoencoder. The method for training an EEG classification model may comprise the steps of: acquiring training data including pairs of EEG signals and classes corresponding to the EEG signals; determining class-specific target data corresponding to each of the classes by performing first phase training on an EEG classification model that receives the training data as an input; and performing second phase training on the EEG classification model that receives the training data as an input, to determine reconstructed data such that a difference between the reconstructed data output by the EEG classification model and the class-specific target data is minimized.

Description

Method and device for training an EEG classification model using a two-phase multi-task autoencoder.

[0001] The present invention relates to a method and device for learning an EEG classification model, and more particularly, to a method and device for learning an EEG classification model using a two-phase multi-task autoencoder.

[0002] Electroencephalography (EEG)-based motion imaging (MI) has potential applications in diverse fields, including rehabilitation, drone control, and virtual reality. However, the subject-dependent nature of EEG signals hinders practical application due to low generalization performance in brain signal decoding.

[0003] The Multitasking Autoencoder (MTAE) technology has improved the generalization performance of brain signal decoding. However, because MTAE uses loss functions with different objectives, it suffers from an imbalance between the loss functions.

[0004] Therefore, a method for learning an EEG classification model without causing an imbalance problem between loss functions is required.

[0005] The present invention provides a method and device for training an EEG classification model by performing first phase learning for determining class-specific target data corresponding to each class and second phase learning for determining reconstructed data output by an EEG classification model so that the difference between the reconstructed data and the class-specific target data is minimized.

[0006] A method for learning an EEG classification model according to an embodiment of the present invention may include the steps of: obtaining learning data composed of pairs of EEG (electroencephalogram) signals and classes corresponding to the EEG signals; performing first phase learning on an EEG classification model that receives the learning data to determine class-specific target data corresponding to each of the classes; and performing second phase learning on an EEG classification model that receives the learning data to determine the reconstructed data such that a difference between reconstructed data output by the EEG classification model and the class-specific target data is minimized.

[0007] The step of determining the target data for each class in the EEG classification model learning method according to one embodiment of the present invention may include the step of classifying and storing data output from the EEG classification model by class according to a label predicted by the EEG classification model; and the step of generating the target data for each class using the data stored by class.

[0008] In the step of determining the target data for each class in the EEG classification model learning method according to one embodiment of the present invention, first phase learning can be performed using a total loss function based on a mean-squared error (MSE) loss function determined according to a difference between an EEG signal included in the learning data and the target data for each class, and a cross-entropy (CE) loss function determined according to a class included in the learning data and a class predicted by the EEG classification model.

[0009] The step of determining the reconstructed data of the EEG classification model learning method according to one embodiment of the present invention may include the step of predicting the class of the EEG signal included in the learning data encoded by the encoder of the EEG classification model; and the step of decoding the EEG signal for which the class is predicted so as to minimize the difference between the target data for each class corresponding to the predicted class and the EEG signal for which the class is predicted, and outputting the reconstructed data.

[0010] The step of determining the reconstructed data of the EEG classification model learning method according to one embodiment of the present invention may perform second phase learning using a total loss function based on a mean square error loss function determined according to a difference between the target data for each class and the reconstructed data and a cross entropy loss function determined according to a class included in the learning data and a class predicted by the EEG classification model.

[0011] An EEG classification model learning device according to one embodiment of the present invention may include an input unit that acquires learning data composed of pairs of EEG signals and classes corresponding to the EEG signals; a first phase learning unit that performs first phase learning on an EEG classification model that receives the learning data to determine class-specific target data corresponding to each of the classes; and a second phase learning unit that performs second phase learning on an EEG classification model that receives the learning data to determine the reconstructed data such that a difference between reconstructed data output by the EEG classification model and the class-specific target data is minimized.

[0012] The first phase learning unit of the EEG classification model learning device according to one embodiment of the present invention can classify and store data output from the EEG classification model by class according to a label predicted by the EEG classification model, and generate target data for each class using the data stored for each class.

[0013] The first phase learning unit of the EEG classification model learning method according to one embodiment of the present invention may perform first phase learning using a total loss function based on a mean square error (MSE) loss function determined according to a difference between an EEG signal included in the learning data and the target data for each class, and a cross entropy (CE) loss function determined according to a class included in the learning data and a class predicted by the EEG classification model.

[0014] The second phase learning unit of the EEG classification model learning method according to one embodiment of the present invention can predict the class of the EEG signal included in the learning data encoded by the encoder of the EEG classification model, and decode the EEG signal for which the class is predicted so that the difference between the target data for each class corresponding to the predicted class and the EEG signal for which the class is predicted is minimized, and output the reconstructed data.

[0015] The second phase learning unit of the EEG classification model learning method according to one embodiment of the present invention can perform second phase learning using a total loss function based on a mean square error loss function determined according to a difference between the target data for each class and the reconstructed data and a cross entropy loss function determined according to a class included in the learning data and a class predicted by the EEG classification model.

[0016] According to one embodiment of the present invention, by training an EEG classification model by performing first phase learning for determining class-specific target data corresponding to each class and second phase learning for determining reconstructed data output by an EEG classification model so as to minimize the difference between the reconstructed data and the class-specific target data, the classification performance for a user's brainwave information can be improved while eliminating learning instability caused by a combination of a loss function that maximizes classification performance and a loss function that minimizes the difference between input data and output data.

[0017] FIG. 1 is a diagram illustrating an EEG classification system including an EEG classification model learning device according to one embodiment of the present invention.

[0018] FIG. 2 is a diagram illustrating an EEG classification model learning device according to an embodiment of the present invention.

[0019] FIG. 3 is an example of the operation of an EEG classification model learning device according to an embodiment of the present invention.

[0020] FIG. 4 is an example of the structure of a decoder of an EEG classification model according to an embodiment of the present invention.

[0021] FIG. 5 is an example of learning and testing operations of an EEG classification model learning device according to an embodiment of the present invention.

[0022] Figure 6 is an example of class-specific target data according to one embodiment of the present invention.

[0023] Figure 7 is an example of visualizing the output of an EEG classification model according to one embodiment of the present invention by phase.

[0024] FIG. 8 is a flowchart illustrating an EEG classification model learning method according to an embodiment of the present invention.

[0025] Hereinafter, embodiments are described in detail with reference to the attached drawings. However, the embodiments may be modified in various ways, and the scope of the patent application is not limited or restricted by these embodiments. It should be understood that all modifications, equivalents, or alternatives to the embodiments are included within the scope of the patent application.

[0026] The terms used in the examples are for illustrative purposes only and should not be construed as limiting. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "comprise" or "have" are intended to indicate the presence of a feature, number, step, operation, component, part, or combination thereof described in the specification, but should be understood to not preclude the presence or addition of one or more other features, numbers, steps, operations, components, parts, or combinations thereof.

[0027] In addition, when describing with reference to the attached drawings, identical components will be assigned the same reference numerals regardless of the drawing numbers, and redundant descriptions thereof will be omitted. When describing embodiments, if a detailed description of a related known technology is judged to unnecessarily obscure the gist of the embodiment, the detailed description will be omitted.

[0028] Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

[0029] FIG. 1 is a diagram illustrating an EEG classification system including an EEG classification model learning device according to one embodiment of the present invention.

[0030] The EEG classification system may be composed of an EEG classification model learning device (120) and an EEG classification device (130).

[0031] An EEG (electroencephalogram) classification model learning device (120) can generate an EEG classification model with a two-phase multi-task auto-encoder structure and learn the EEG classification model using learning data stored in a learning database (110). Specifically, the EEG classification model learning device (120) can perform first phase learning to determine class-specific target data corresponding to each class and second phase learning to determine reconstructed data output by the EEG classification model so that the difference between the reconstructed data and the class-specific target data is minimized.

[0032] The specific configuration and operation of the EEG classification model learning device (120) are described in detail with reference to FIG. 2 below.

[0033] The EEG classification model learning device (120) can perform first phase learning and second phase learning and transmit the determined EEG classification model (101) to the EEG classification device (130).

[0034] The EEG classification device (130) can input the user's brain wave information (102) measured from the user into the EEG classification model (101) and output a class corresponding to the movement the user is thinking of as an EEG classification result (103).

[0035] The EEG classification model learning device (120) trains the EEG classification model by performing first phase learning to determine class-specific target data corresponding to each class and second phase learning to determine reconstructed data output by the EEG classification model so as to minimize the difference between the reconstructed data and the class-specific target data, thereby improving the classification performance for brainwave information (102) while eliminating learning instability caused by the combination of a loss function that maximizes classification performance and a loss function that minimizes the difference between input data and output data.

[0036] FIG. 2 is a diagram illustrating an EEG classification model learning device according to an embodiment of the present invention.

[0037] The EEG classification model learning device (120) may include an input unit (210), a first pace learning unit (220), a second pace learning unit (230), and a classification model determination unit (240), as illustrated in FIG. 2. At this time, the first pace learning unit (220), the second pace learning unit (230), and the classification model determination unit (240) may be different processors, as illustrated in FIG. 2, or may be individual modules included in a program executed in one processor.

[0038] The input unit (210) can obtain learning data consisting of pairs of EEG signals and classes corresponding to the EEG signals. At this time, the input unit (210) can obtain learning data from the learning database (110) or receive learning data from a user. For example, the input unit (210) can be a communication interface or communication port connected to the learning database (110) by wire or wirelessly, or an input interface for receiving learning data from a user.

[0039] The first phase learning unit (220) can input training data into an EEG classification model generated using a two-phase multi-task auto-encoder structure. Furthermore, the first phase learning unit (220) can perform first phase learning on the EEG classification model to determine class-specific target data corresponding to each class. Specifically, the first phase learning unit (220) can classify and store data output from the EEG classification model into classes based on labels predicted by the EEG classification model, and generate class-specific target data using the data stored for each class.

[0040] In addition, the first phase learning unit (220) can perform first phase learning using a total loss function based on a mean-squared error (MSE) loss function determined based on the difference between the EEG signal included in the learning data and the target data for each class, and a cross-entropy (CE) loss function determined based on the class included in the learning data and the class predicted by the EEG classification model.

[0041] The second phase learning unit (230) can input training data into the EEG classification model. Then, the second phase learning unit (230) can perform second phase learning on the EEG classification model to determine the reconstructed data so as to minimize the difference between the reconstructed data output by the EEG classification model and the target data for each class. Specifically, the second phase learning unit (230) can predict the class of the EEG signal included in the training data encoded by the encoder of the EEG classification model, and decode the EEG signal for which the class is predicted so as to minimize the difference between the target data for each class corresponding to the predicted class and the EEG signal for which the class is predicted, and output the reconstructed data.

[0042] In addition, the second phase learning unit (230) can perform second phase learning using a total loss function based on a mean square error loss function determined based on the difference between the target data and the reconstructed data for each class and a cross entropy loss function determined based on the class included in the learning data and the class predicted by the EEG classification model.

[0043] The classification model determination unit (240) can evaluate the performance of the EEG classification model that has performed the first phase learning and the second phase learning. If the performance of the EEG classification model that has performed the first phase learning and the second phase learning exceeds a threshold, the classification model determination unit (240) can determine the EEG classification model as the final EEG classification model and output it or transmit it to the EEG classification device (130).

[0044] If the performance of the EEG classification model that performed the first phase learning and the second phase learning is below a threshold, the classification model determination unit (240) may request the first phase learning unit (220) and the second phase learning unit (230) to repeatedly perform the first phase learning and the second phase learning until the performance of the EEG classification model that performed the first phase learning and the second phase learning exceeds the threshold.

[0045] FIG. 3 is an example of the operation of an EEG classification model learning device according to an embodiment of the present invention.

[0046] The training data input to the EEG classification model can be composed of pairs of EEG signals and classes corresponding to the EEG signals. In this case, the pair of X, which is an EEG signal, and y, which is a class label indicating the class, is can be defined as. Also, , can be. And, N is the number of EEG signals included in the training data, C is the number of electrodes used for EEG measurement, S is the number of sample points measured from each electrode, and L can be the number of classes.

[0047] Even EEG signals with the same class label can vary across subjects due to physiological and anatomical differences in the brain. Therefore, the training data is D = {D1, D2, ,,, D M} can be defined as, where M is the number of subjects. The subject independent classification is D corresponding to test subject j. j The data of is used for testing and other subjects D corresponding to j≠i i The data can be used for learning.

[0048] This could be a model that integrates an algorithm that showed state-of-the-art performance on the MI (motor imagery) dataset as an encoder to maximize the ability to improve the classification performance of the model by effectively extracting discriminant features from the EEG classification model.

[0049] In step (310), the first phase learning unit (220) may perform first phase learning on the EEG classification model to determine class-specific target data corresponding to each class. For example, the first phase learning unit (220) may determine the total loss function L defined as shown in mathematical expression 1. total First-phase learning can be performed using .

[0050] [Mathematical Formula 1]

[0051]

[0052] At this time, L CE is a cross entropy (CE) loss function determined by the class included in the learning data and the class predicted by the EEG classification model, and can be defined as in mathematical expression 2.

[0053] [Equation 2]

[0054]

[0055] At this time, p(y i ′) = Softmax(z1).

[0056] L MSE is a mean square error (MSE) loss function determined by the difference between the EEG signal included in the learning data and the class-specific target data, and can be defined as in mathematical expression 3.

[0057] [Equation 3]

[0058]

[0059] At this time, may be a Frobenius norm.

[0060] Also, the loss function L CE The loss function weight parameter is Wow, loss function L MSE The loss function weight parameter is It determines the contribution of each loss function during the learning process, and the optimal value can be obtained through hyperparameter optimization.

[0061] After the first phase learning is completed, the first phase learning unit (220) converts the data X' output from the EEG classification model into the label y' predicted from the EEG classification model. i According to the above, the first face learning unit (220) can classify and store X's predicted as a class corresponding to the movement of the right hand into the Right hand group and store them. In addition, the first face learning unit (220) can classify and store X's predicted as a class corresponding to the movement of the left hand into the Left hand group and store them. In addition, the first face learning unit (220) can classify and store X's predicted as a class corresponding to the movement of both feet into the Both feet group and store them. In addition, the first face learning unit (220) can classify and store X's predicted as a class corresponding to the movement of the tongue into the Tongue group and store them.

[0062] At this time, each group has t for k=1, 2, ... L k is represented as , where k can correspond to a class. For example, y' i If is predicted as class 1, the first face learning unit (220) X' i can be stored in t1.

[0063] And, the first face learning unit (220) can generate target data for each class using data stored for each class. For example, the first face learning unit (220) can generate target data for each t k The average of all data included can be generated as class-specific target data representing each class.

[0064] That is, T, the target data for each class, is defined as, is t k It can be the average of . In addition, the first phase learning unit (220) can perform first phase learning only on the learning data so that the test data is not used to generate class-specific target data. And, T is the predicted label y' i Since it is calculated based on the first face learning unit (220), the EEG classification model can be learned in a direction that maximizes the classification performance of the learning data to increase the representativeness of T for each class.

[0065] In step (320), the second phase learning unit (230) may perform second phase learning on the EEG classification model to determine the reconstructed data so that the difference between the reconstructed data output by the EEG classification model and the target data for each class is minimized. For example, the second phase learning unit (230) may determine the total loss function L defined as shown in Equation 4. total Second phase learning can be performed using .

[0066] [Equation 4]

[0067]

[0068] At this time, the second phase learning unit (230) is the reconstruction data X'' outputted by the EEG classification model and the class-specific target data of class y. Using L MSE can be decided. At this time, can be selected based on the actual label y included in the training data in the class-specific target data T. For example, L MSE can be defined as in mathematical formula 5.

[0069] [Equation 5]

[0070]

[0071] As a result, L MSE L uses the class y'' predicted in the second phase learning. CE It can perform a similar purpose. In addition, the second face learning unit (230) is defined as L in mathematical expression 6. CE can be used.

[0072] [Equation 6]

[0073]

[0074] At this time, p(y i '') = Softmax(z2).

[0075] That is, the total loss function L used by the second phase learning unit (230) total Silver L CE L modified to perform similar purposes MSE By including L CE Wow L MSE The weight parameters of the second phase are relieved of the conflict between the two and Sensitivity to may be reduced.

[0076] However, L CE Wow L MSE The influence of the second phase learning unit (230) on learning may be different. Therefore, the second phase weight parameter and Using L CE Wow L MSE can be adjusted. For example, L CE Weight parameters of Depending on the change, the performance of the EEG classification model may change as shown in Table 1.

[0077] [Table 1]

[0078]

[0079] At this time, L MSE Since L is calculated based on the class-specific target data determined in the first phase learning, CE It has a considerably lower scale compared to . Therefore, may be fixed at 1.0. Also, in Table 1, * indicates the best performance. In both datasets, The highest accuracy and F1-score were achieved when the value was 0.01.

[0080] Additionally, the first-phase learning and the second-phase learning can be implemented sequentially as end-to-end learning.

[0081] FIG. 4 is an example of the structure of a decoder of an EEG classification model according to an embodiment of the present invention.

[0082] The EEG classification model may be an autoencoder structure including an encoder and a decoder, as illustrated in Fig. 3. At this time, the output of the decoder of the EEG classification model must be identical to the input data shape of the encoder of the EEG classification model.

[0083] Accordingly, the latent vector (410) output from the encoder of the EEG classification model can be converted into a vector of the same size as the last layer of the encoder by passing through the FC layer (420) of the decoder (430). Then, the output of the FC layer (420) can be converted into a tensor of an appropriate size by passing through the reshape layer (430) three times, and then can be output by expanding the size of the feature map.

[0084] FIG. 5 is an example of learning and testing operations of an EEG classification model learning device according to an embodiment of the present invention.

[0085] The BCI competition IV 2a (BCIC-IV2a)(510), a representative MI classification benchmark provided by Graz University of Technology, is a dataset that is difficult to decode because it is performed in an uncontrolled environment. It consists of 5,184 trials measured across two sessions by nine subjects for four MI tasks: left hand, right hand, both feet, and tongue. The experiment was conducted with a 2-second screen viewing period, 4-second MI, and 1.25-second rest period. In each trial, EEG data were recorded at a sampling rate of 250 Hz from 22 electrodes (Fz, FC3, FC1, FCz, FC2, FC4, C5, C3, C1, Cz, C2, C4, C6, CP3, CP1, CPz, CP2, CP4, P1, Pz, P2, POz). The data were segmented into 4.5-second segments starting 0.5 seconds before the onset of MI, and the structure is as follows: It was used like this.

[0086] The MI benchmark OpenBMI (520) is a dataset generated by recording EEG signals at a sampling rate of 1000 Hz from 54 subjects for left-hand and right-hand MI tasks using 62 electrodes. The experiment consisted of four sessions, with each session performing 50 repetitions on the left hand and 50 repetitions on the right hand. The data were segmented into 4-second segments during MI, and the signals were downsampled to 100 Hz. 20 electrodes (FC5, FC3, FC1, FC2, FC4, FC6, C5, C3, C1, Cz, C2, C4, C6, CP5, CP3, CP1, CPz, CP2, CP4, CP6) were selected. The data structure is as follows: It was used like this.

[0087] BCI competition IV 2a (BCIC-IV2a) (510) and MI benchmark OpenBMI (520) may be composed of learning data (502) used for learning and test data (501) used for testing.

[0088] The training data (502) of BCI competition IV 2a (BCIC-IV2a) (510) or MI benchmark OpenBMI (520) can be input to the EEG classification model training device (120) as input data (530) on which both first-phase learning and second-phase learning are performed.

[0089] At this time, the EEG classification model learning device (120) may perform first phase learning (540) after preprocessing the input data (530) to determine class-specific target data corresponding to each class. Next, the EEG classification model learning device (120) may perform second phase learning (550) to determine reconstructed data so that the difference between the reconstructed data output by the EEG classification model and the class-specific target data is minimized.

[0090] In addition, the EEG classification model learning device (120) can measure the performance of the EEG classification model using test data (501). At this time, the EEG classification model learning device (120) can preprocess input data (560) composed of test data (501) and then input the preprocessed data into the EEG classification model on which the second phase learning (550) has been completed. In addition, the EEG classification model learning device (120) can measure the performance of the EEG classification model on which the second phase learning (550) has been completed by comparing the labels included in the reconstructed data output by the EEG classification model on which the second phase learning (550) has been completed with the labels included in the test data (501).

[0091] Figure 6 is an example of class-specific target data according to one embodiment of the present invention.

[0092] In AE architecture, the size of the latent vector plays a critical role in reconstruction performance.

[0093] A larger latent vector stores more information from the input data, improving reconstruction accuracy but potentially increasing generalizability and noise. Conversely, a smaller latent vector forces AE to extract only the most salient features of the input data, potentially lowering reconstruction accuracy but reducing generalizability and noise.

[0094] The EEG classification model learning device (120) can set the size of the latent vector to the number of classes, allowing the encoder to learn the most discriminative high-level features for each class. Accordingly, reconstructed data belonging to the same class may exhibit similar patterns, while data from different classes may exhibit distinct differences.

[0095] For example, the class-specific target data determined in the first phase learning using TCNet-Fusion as an encoder for each of the four classes included in the Cz channel of the BCI-IV2a dataset may be as shown in Fig. 6.

[0096] Figure 7 is an example of visualizing the output of an EEG classification model according to one embodiment of the present invention by phase.

[0097] Image (710) is an example of a latent vector output by the first phase learning unit (220) of the EEG classification model learning device (120) for the BCIC-IV2a dataset, visualized using t-SNE (t-distributed stochastic neighbor embedding), and image (720) is an example of a latent vector output by the second phase learning unit (230) of the EEG classification model learning device (120) for the BCIC-IV2a dataset, visualized using t-SNE (t-distributed stochastic neighbor embedding).

[0098] In image (710), latent vectors of classes 1 to 4 may be unclassified. On the other hand, in image (720), latent vectors of classes 1 to 4 may be classified in different directions. In particular, test data not used for learning may also be classified by class, as illustrated in image (720).

[0099] FIG. 8 is a flowchart illustrating an EEG classification model learning method according to an embodiment of the present invention.

[0100] In step (810), the input unit (210) can obtain learning data consisting of pairs of EEG signals and classes corresponding to the EEG signals.

[0101] In step (820), the first phase learning unit (220) can input learning data into an EEG classification model generated with a two-phase multi-task auto-encoder structure. Then, the first phase learning unit (220) can perform first phase learning on the EEG classification model to determine class-specific target data corresponding to each class. Specifically, the first phase learning unit (220) can classify and store data output from the EEG classification model by class according to labels predicted by the EEG classification model, and generate class-specific target data using the data stored by class.

[0102] In step (830), the second phase learning unit (230) can input training data into the EEG classification model. Then, the second phase learning unit (230) can perform second phase learning on the EEG classification model to determine the reconstructed data so as to minimize the difference between the reconstructed data output by the EEG classification model and the target data for each class. Specifically, the second phase learning unit (230) can predict the class of the EEG signal included in the training data encoded by the encoder of the EEG classification model, and decode the EEG signal for which the class is predicted so as to minimize the difference between the target data for each class corresponding to the predicted class and the EEG signal for which the class is predicted, and output the reconstructed data.

[0103] In step (840), the classification model determination unit (240) can evaluate the performance of the EEG classification model that performed the first phase learning and the second phase learning. In addition, the classification model determination unit (240) can check whether the performance of the evaluated EEG classification model exceeds a threshold value. If the performance of the EEG classification model that performed the first phase learning and the second phase learning exceeds the threshold value, the classification model determination unit (240) can perform step (850). If the performance of the EEG classification model that performed the first phase learning and the second phase learning is lower than the threshold value, the classification model determination unit (240) can request the first phase learning unit (220) and the second phase learning unit (230) to repeatedly perform steps (820) to (830) until the performance of the EEG classification model that performed the first phase learning and the second phase learning exceeds the threshold value.

[0104] In step (850), the classification model determination unit (240) can determine the EEG classification model as the final EEG classification model and output it or transmit it to the EEG classification device (130).

[0105] The present invention trains an EEG classification model by performing first phase learning for determining class-specific target data corresponding to each class and second phase learning for determining reconstructed data output by an EEG classification model so as to minimize the difference between the reconstructed data and the class-specific target data, thereby improving the classification performance for a user's brainwave information while eliminating learning instability caused by a combination of a loss function that maximizes classification performance and a loss function that minimizes the difference between input data and output data.

[0106] The present invention was created in accordance with the following tasks.

[0107] [Project ID] 1711185888

[0108] [Assignment Number] 2020M3C1B8081320

[0109] [Ministry Name] Ministry of Science and ICT

[0110] [Name of Project Management (Specialist) Institution] National Research Foundation of Korea

[0111] [Research Project Name] Human Plus Convergence Research and Development Challenge Project

[0112] [Research Project Name] Brain Stimulation-EEG Synchronization Analysis and Feedback Control Fusion Platform

[0113] development

[0114] [Contribution rate] 1 / 2

[0115] [Name of Project Performing Organization] Seoul National University of Science and Technology

[0116] [Research Period] January 1, 2023 - December 31, 2023

[0117] [National Research and Development Project Supporting This Invention]

[0118] [Project ID] 1711194684

[0119] [Assignment Number] 00208492

[0120] [Ministry Name] Ministry of Science and ICT

[0121] [Name of Project Management (Specialist) Institution] National Research Foundation of Korea

[0122] [Research Project Name] Individual Basic Research (Ministry of Science and ICT)

[0123] [Research Project Title] AI-Based Postoperative Delirium Using Multichannel EEG

[0124] Development of early diagnosis technology

[0125] [Contribution rate] 1 / 2

[0126] [Name of the project performing organization] Seoul National University of Science and Technology Industry-Academic Cooperation Foundation

[0127] [Research Period] March 1, 2023 - February 29, 2024

[0128] Meanwhile, the EEG classification model learning device or EEG classification model learning method based on speech data according to the present invention can be written as a program that can be executed on a computer and can be implemented in various recording media such as a magnetic storage medium, an optical reading medium, and a digital storage medium.

[0129] Implementations of the various technologies described herein may be implemented as digital electronic circuitry, or as computer hardware, firmware, software, or combinations thereof. Implementations may be implemented as a computer program product, for example, a computer program tangibly embodied in a machine-readable storage device (computer-readable medium), for processing by the operation of a data processing device, for example, a programmable processor, a computer, or multiple computers, or for controlling the operation thereof. A computer program, such as the computer program(s) described above, may be written in any form of programming language, including compiled or interpreted languages, and may be deployed in any form, including as a standalone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program may be deployed to be processed on one computer or multiple computers at a single site, or to be distributed across multiple sites and interconnected by a communications network.

[0130] Processors suitable for processing a computer program include, for example, both general-purpose and special-purpose microprocessors, and any one or more processors of any type of digital computer. Typically, a processor will receive instructions and data from read-only memory or random-access memory, or both. Components of a computer may include at least one processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer may include, or be coupled to receive data from, transmit data to, or both, one or more mass storage devices, such as magnetic, magneto-optical, or optical disks, for storing data. Information carriers suitable for embodying computer program instructions and data include, for example, semiconductor memory devices, magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as compact disk read only memory (CD-ROM), digital video disks (DVD), magneto-optical media such as floptical disks, read only memory (ROM), random access memory (RAM), flash memory, erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), etc. The processor and memory may be supplemented by, or included in, special purpose logic circuitry.

[0131] Additionally, the computer-readable medium may be any available medium that can be accessed by a computer, and may include all computer storage media.

[0132] While this specification contains details of a number of specific implementations, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be unique to particular embodiments of particular inventions. Certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although features may operate in a particular combination and may initially be described as being claimed as such, one or more features from a claimed combination may in some cases be excluded from that combination, and the claimed combination may be modified into a subcombination or variation of a subcombination.

[0133] Likewise, while operations are depicted in the drawings in a particular order, this should not be construed as requiring that those operations be performed in the particular or sequential order depicted to achieve desired results, or that all depicted operations be performed. In certain instances, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various device components of the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the program components and devices described may generally be integrated together in a single software product or packaged into multiple software products.

[0134] Meanwhile, the embodiments of the present invention disclosed in this specification and drawings are merely specific examples presented to aid understanding and are not intended to limit the scope of the present invention. It will be apparent to those skilled in the art that other modifications based on the technical concepts of the present invention are possible in addition to the embodiments disclosed herein.

Claims

1. A step of obtaining learning data consisting of pairs of EEG (electroencephalogram) signals and classes corresponding to the EEG signals; A step of performing first phase learning on an EEG classification model that receives the above learning data to determine class-specific target data corresponding to each of the above classes; and A step of performing second phase learning on an EEG classification model that receives the above learning data, thereby determining the reconstructed data so that the difference between the reconstructed data output by the EEG classification model and the target data for each class is minimized. A method for learning an EEG classification model including:

2. In paragraph 1, The step of determining the target data for each class is as follows: A step of classifying and storing data output from the EEG classification model into classes according to labels predicted by the EEG classification model; and A step for generating target data for each class using data saved for each class. A method for learning an EEG classification model including:

3. In paragraph 1, The step of determining the target data for each class is as follows: An EEG classification model learning method that performs first-phase learning using a total loss function based on a mean-squared error (MSE) loss function determined according to the difference between the EEG signal included in the above learning data and the target data for each class, and a cross-entropy (CE) loss function determined according to the class included in the above learning data and the class predicted by the EEG classification model.

4. In paragraph 1, The step of determining the above reconstruction data is: A step of predicting the class of the EEG signal included in the encoded training data in the encoder of the above EEG classification model; and A step of decoding the EEG signal that predicted a class so that the difference between the target data for each class corresponding to the predicted class and the EEG signal that predicted the class is minimized, and outputting the reconstructed data. A method for learning an EEG classification model including:

5. In paragraph 1, The step of determining the above reconstruction data is: A method for learning an EEG classification model, which performs second phase learning using a total loss function based on a mean square error loss function determined according to the difference between the target data for each class and the reconstructed data and a cross entropy loss function determined according to the class included in the learning data and the class predicted by the EEG classification model.

6. An input unit for obtaining learning data consisting of pairs of EEG signals and classes corresponding to the EEG signals; A first phase learning unit that performs first phase learning on an EEG classification model that receives the above learning data and determines class-specific target data corresponding to each of the above classes; and A second phase learning unit that performs second phase learning on an EEG classification model that receives the above learning data and determines the reconstructed data so that the difference between the reconstructed data output by the EEG classification model and the target data for each class is minimized. An EEG classification model learning device comprising:

7. In paragraph 6, The above first phase learning unit, An EEG classification model learning device that classifies data output from the EEG classification model into classes according to labels predicted by the EEG classification model and stores the data, and generates target data for each class using the data stored for each class.

8. In paragraph 6, The above first phase learning unit, An EEG classification model learning device that performs first phase learning using a total loss function based on a mean square error (MSE) loss function determined according to the difference between the EEG signal included in the learning data and the target data for each class, and a cross entropy (CE) loss function determined according to the class included in the learning data and the class predicted by the EEG classification model.

9. In paragraph 6, The above second phase learning unit, An EEG classification model learning device that predicts the class of an EEG signal included in encoded learning data in an encoder of the above EEG classification model, decodes the EEG signal for which the class has been predicted so that the difference between the target data for each class corresponding to the predicted class and the EEG signal for which the class has been predicted is minimized, and outputs the reconstructed data.

10. In paragraph 6, The above second phase learning unit, An EEG classification model learning device that performs second phase learning using a total loss function based on a mean square error loss function determined according to the difference between the target data for each class and the reconstructed data and a cross entropy loss function determined according to the class included in the learning data and the class predicted by the EEG classification model.

Images