EEG emotion recognition and emotional disorder detection method and device based on multi-task learning, equipment and medium

Abstract

The invention discloses an EEG (electroencephalogram) emotion recognition and emotional disorder detection method and device based on multi-task learning, equipment and a medium, and relates to the field of electroencephalogram signal emotion and emotional disorder recognition in the field of feature recognition. Allocating a corresponding expert weight to each task through a gating network, and constructing a neural network model; inputting the electroencephalogram data after the expert weight is calculated into a tower layer module in the neural network model for task classification to obtain a classification result of each task; allocating an uncertainty weight to each task in the electroencephalogram data, dynamically adjusting a task loss weight through a dynamic weighting loss function, reconstructing a neural network model, and determining an optimized neural network model; and inputting the electroencephalogram data into the optimized neural network model, and determining a task classification result, so that the problem of data scarcity is solved, shared information can be fully utilized, and emotion recognition and emotional disorder detection are realized.

Description

Technical Field

[0001] This application relates to the field of electroencephalogram (EEG) emotion and emotion disorder recognition in the field of biometric recognition, and particularly relates to a method, device, equipment and medium for EEG emotion recognition and emotion disorder detection based on multi-task learning. Background Technique

[0002] Emotion is an important part of human experience and has a profound impact on behavior. Usually, emotions are transient and adaptive, which can help individuals effectively cope with external stimuli. However, when the emotional state loses its regulatory ability or persists abnormally, it may develop into an emotion disorder, such as Major Depressive Disorder (MDD), bipolar disorder, and generalized anxiety disorder. Emotion disorders not only manifest as disorders of emotion regulation, but also significantly affect an individual's cognitive function, autonomic nervous function, and stress response.

[0003] Research shows that different emotional states will lead to significant differences in the electrocortical activity of the brain, reflecting the dynamic changes of emotions. Electroencephalogram (EEG), due to its characteristics of low cost, non-invasiveness, and high temporal resolution, is widely used in emotion recognition and emotion disorder detection. EEG can capture the electrical activity of the central nervous system in real time and reflect the physiological characteristic changes of the emotional state. For example, different emotional states will lead to changes in features such as Power Spectral Density (PSD), Differential Entropy (DE), and frequency domain asymmetry.

[0004] Remarkable progress has been made in the research of emotion recognition based on EEG. Traditional methods such as machine learning techniques like Support Vector Machine (SVM) can perform emotion classification based on the extracted EEG features. With the development of deep learning, models such as Convolutional Neural Network (CNN), Temporal-Spatial Convolutional Network (TSception), and Graph Convolutional Network (Gateway Connection Network, GCN) have performed well in emotion recognition tasks. For example, a Dynamic Graph CNN (DGCNN) based on graphs optimizes the relationship representation between nodes by dynamically updating the adjacency matrix, improving the accuracy of emotion recognition.

[0005] However, most current EEG-based studies adopt the single-task learning (Standard Template Library, STL) method, which focuses on the optimization of a single task, such as emotion classification or mood disorder detection. This method has the following limitations:

[0006] ①Insufficient utilization of shared information: Single-task learning ignores the shared information between multiple related tasks and cannot effectively capture the potential connection between emotions and mood disorders.

[0007] ②Data scarcity problem: The high cost of data annotation in the field of mood disorder detection results in insufficient training data, which limits the generalization ability of the model. Summary of the Invention

[0008] The purpose of this application is to provide an EEG emotion recognition and mood disorder detection method, device, equipment and medium based on multi-task learning, which solves the problems of insufficient utilization of shared information and data scarcity.

[0009] To achieve the above purpose, this application provides the following solutions:

[0010] In the first aspect, this application provides an EEG emotion recognition and mood disorder detection method based on multi-task learning, including:

[0011] Extract the task-shared features of EEG data according to the heterogeneous expert network.

[0012] Based on the task-shared features, assign corresponding expert weights to each task through the gating network, and use the historical EEG data as the input and the historical task classification results as the output to construct a neural network model.

[0013] Input the EEG data after calculating the expert weights into the tower layer module of the neural network model for task classification to obtain the classification results of each task.

[0014] Based on the neural network model, assign uncertainty weights to each task in the EEG data, and through the dynamic weighted loss function, dynamically adjust the task loss weights according to the tasks after assigning uncertainty weights.

[0015] According to the adjusted task loss weights, correct the bias of the neural network model, reconstruct the neural network model, and determine the optimized neural network model.

[0016] Input the EEG data into the optimized neural network model to determine the task classification results; the task classification results include emotion state detection and mood disorder detection; the emotion state detection includes positive emotion, negative emotion and neutral emotion; the mood disorder detection task includes normal mood and mood disorder.

[0017] In a second aspect, the present application provides an EEG emotion recognition and emotion disorder detection device based on multi-task learning, including:

[0018] A task-sharing feature extraction module, where a heterogeneous expert network extracts task-sharing features of EEG data.

[0019] A neural network model construction module, based on the task-sharing features, assigns corresponding expert weights to each task through a gating network, and uses historical EEG data as input and historical task classification results as output to construct a neural network model.

[0020] A task loss weight determination module, inputs the EEG data after calculating the expert weights into the tower layer module of the neural network model for task classification to obtain the classification results of each task.

[0021] A task loss weight adjustment module, based on the neural network model, assigns uncertainty weights to each task in the EEG data, and through a dynamic weighted loss function, dynamically adjusts the task loss weights according to the tasks after assigning uncertainty weights.

[0022] An optimized neural network model determination module, according to the adjusted task loss weights, corrects the bias of the neural network model, reconstructs the neural network model, and determines the optimized neural network model; inputs the EEG data into the optimized neural network model to determine the task classification results.

[0023] In a third aspect, the present application provides a computer device, including: a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the steps of the EEG emotion recognition and emotion disorder detection method based on multi-task learning described in any one of the above.

[0024] In a fourth aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the EEG emotion recognition and emotion disorder detection method based on multi-task learning described in any one of the above.

[0025] According to the specific embodiments provided by the present application, the present application has the following technical effects:

[0026] The present application provides a method, apparatus, device and medium for EEG emotion recognition and emotion disorder detection based on multi-task learning. Task-shared features of EEG data are extracted according to a heterogeneous expert network. Based on the task-shared features, a gating network assigns corresponding expert weights to each task, and a neural network model is constructed with historical EEG data as the input and historical task classification results as the output, realizing the joint detection of emotional states and emotion disorders. The EEG data after calculating the expert weights is input into a tower layer module for task classification to obtain the classification results of each task. Based on the neural network model, an uncertainty weight is assigned to each task in the EEG data, and through a dynamic weighted loss function, according to the tasks after assigning the uncertainty weights, the task loss weights are adjusted. The model can learn specific representations for different tasks, thereby improving performance in multi-task learning scenarios. According to the adjusted task loss weights, the bias of the neural network model is corrected, the neural network model is reconstructed, and an optimized neural network model is determined. The dynamic weighted loss function can adaptively adjust the task loss weights according to the uncertainty of the tasks, which helps the model better balance the learning difficulty and speed of different tasks. The EEG data is input into the optimized neural network model to determine the task classification results. Based on the optimized model, the emotional states and emotion disorders of the EEG data are identified, and the classification results are output. Based on the optimized model, the emotional states and emotion disorders of the input EEG data are identified, and the classification results are output, significantly improving the accuracy and generalization ability of emotion and emotion disorder recognition, and revealing the potential connection between emotion and higher cognitive processes. First, the present application uses historical EEG data as the input of the neural network model, and this historical EEG data contains rich emotional state and emotion disorder information, solving the problem of data scarcity. Second, a heterogeneous expert network is used to extract task-shared features from the EEG data, and these shared features are information shared by multiple tasks, reducing the amount of data required for each task to learn alone, and solving the problem of insufficient utilization of shared information. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0028] Figure 1 It is a schematic flowchart provided by an embodiment of the present application;

[0029] Figure 2 It is a schematic working diagram provided by an embodiment of the present application;

[0030] Figure 3The structural diagram of the gating mechanism provided by the embodiment of the present application;

[0031] Figure 4 The structural diagram of the tower layer network provided by the embodiment of the present application;

[0032] Figure 5 The flow chart of the dataset paradigm design provided by the embodiment of the present application;

[0033] Figure 6 The schematic diagram of the subject division provided by the embodiment of the present application;

[0034] Figure 7 The structural schematic diagram of a computer device provided by an embodiment of the present application. Detailed implementation manners

[0035] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0036] To make the above objects, features, and advantages of the present application more obvious and understandable, the present application will be further described in detail below in conjunction with the accompanying drawings and specific implementation manners.

[0037] Such as Figure 2As shown in the figure, this application discloses a method for EEG emotion recognition and emotion disorder detection based on multi-task learning. First, collect EEG signals, perform band-pass filtering and independent component analysis (ICA) denoising processing, and extract differential entropy (DE) features to convert the three-dimensional EEG time series signals into two-dimensional feature matrices. Secondly, construct an MMoGCN model, including a heterogeneous expert network, a gating mechanism, a tower layer module, and a dynamic weighted loss function. The heterogeneous expert network combines a multi-scale feature reconstruction graph convolutional network (MSFR-GCN) and a graph convolutional network based on feature adaptive graph connection (Gateway Connection Network, GCN) to extract task-shared features. The gating mechanism realizes the dynamic allocation of task weights through the Softmax function. The tower layer module uses a three-layer fully connected network to map the weighted output to a task-specific class space. The dynamic weighted loss function adjusts the task loss weights according to the uncertainty of the tasks to optimize the model performance. Finally, use the trained model to jointly identify the emotional state and emotional disorders of the EEG data and output the classification results. This application effectively improves the accuracy and generalization ability of emotion and emotion disorder recognition, and has the advantages of high computational efficiency, low time complexity, and strong robustness.

[0038] MMoGCN combines four modules: an expert module, a gating mechanism, a tower layer, and an automatically weighted loss function. The expert module uses a heterogeneous expert network to extract task-shared features. The gating network is responsible for assigning different expert weights to each task to achieve dynamic task allocation. Use Adaptive Weighted Loss (AWL) to model the uncertainty weights for different classification tasks respectively and introduce a dynamic loss. Finally, the tower layer maps the weighted expert output to a task-specific class space.

[0039] As Figure 1 shown in the figure, an embodiment of this application provides a method for EEG emotion recognition and emotion disorder detection based on multi-task learning, specifically including:

[0040] S1: Extract task-shared features of EEG data according to the heterogeneous expert network.

[0041] S2: Based on the task-shared features, assign corresponding expert weights to each task through the gating network, and construct a neural network model with historical EEG data as the input and historical task classification results as the output.

[0042] S3: Input the EEG data after calculating the expert weights into the tower layer module of the neural network model for task classification to obtain the classification results of each task.

[0043] S4: Based on the neural network model, assign uncertainty weights to each task in the EEG data, and through a dynamic weighted loss function, dynamically adjust the task loss weights according to the tasks with assigned uncertainty weights.

[0044] S5: According to the adjusted task loss weights, correct the bias of the neural network model, reconstruct the neural network model, and determine the optimized neural network model.

[0045] S6: Input the EEG data into the optimized neural network model to determine the task classification results; the task classification results include emotional state detection and emotional disorder detection; the emotional state detection includes positive emotion, negative emotion, and neutral emotion; the emotional disorder detection includes normal emotion and emotional disorder.

[0046] Multi-task learning provides a new idea for solving the technical problems of this application. MTL can utilize the shared information between tasks by simultaneously optimizing multiple related tasks, improving the performance and generalization ability of the model. In the field of EEG research, multi-task learning has demonstrated significant advantages. For example, some research has simultaneously performed emotion recognition and context information classification through a multi-task framework, significantly improving the accuracy of emotion classification. In addition, deep learning models based on task sharing mechanisms, such as IDMMOE and multi-task encoder networks based on SincNet, can effectively mine the correlations between emotion dimensions, further enhancing the effect of cross-subject emotion recognition.

[0047] In addition, previous EEG datasets mainly focused on emotion recognition tasks of healthy individuals and lacked data on patients with emotional disorders. However, the emotional responses of patients with emotional disorders have important research significance, which helps to reveal the neural mechanisms of emotional disorders and provides new methods for the detection and intervention of emotional disorders.

[0048] Therefore, this application proposes an EEG emotion recognition and emotional disorder detection model based on multi-task learning - the multi-gate mixture-of-experts graph convolutional network (MMoGCN). This model extracts shared features between tasks through a shared expert network, combines a dynamic gating mechanism and an adaptive weighted loss to balance the task learning difficulty, thereby achieving the joint detection of emotional states and emotional disorders. In addition, this application designs an EEG dataset containing normal people and patients with emotional disorders, and records their EEG performances under different emotional stimuli through a specific experimental paradigm.

[0049] Through a multi-task learning framework, this application breaks through the limitations of single-task learning and effectively improves the performance and generalization ability of the model by leveraging the shared information between emotions and mood disorders, where mood disorders include major depressive disorder, bipolar disorder, and anxiety disorder. Meanwhile, the proposed model demonstrates excellent performance in cross-subject emotion classification and mood disorder detection tasks, providing a new solution for EEG-based emotion recognition and mood disorder detection.

[0050] Further, in an exemplary embodiment, step S1 can be replaced by the following steps.

[0051] S101: Learn the EEG data using an adaptive graph learning method and construct a graph convolutional network based on feature-adaptive graph connections.

[0052] To more effectively utilize emotional stimuli to evaluate the differences in cognitive states between normal individuals and patients with mood disorders, this application designs the 3-Back task and mental arithmetic task, providing important insights into the brain functions related to higher cognitive processes. In the experiment, each participant needs to watch 10 movie clips (covering three different emotions) and immediately complete the corresponding cognitive task test after watching each clip. After watching the first 5 clips, the participant completes the 3-Back task test after each clip; after watching the last 5 clips, the participant completes the mental arithmetic task test after each clip. Meanwhile, a 5-minute break is set between the front and back tasks.

[0053] Collect EEG signals through an EEG acquisition device, perform band-pass filtering, and use Independent Component Analysis (ICA) to remove artifacts, and extract Differential Entropy (DE) features from the preprocessed EEG signals.

[0054] Utilize a heterogeneous expert network, including a graph convolutional network with multi-scale feature reconstruction (MSFR-GCN) and a graph convolutional network based on feature-adaptive graph connections (GCN), to extract task-shared features.

[0055] The datasets used in this application to verify the model performance come from the self-designed dataset (DCEMD) and the public dataset DEAP.

[0056] For the DCEMD dataset, (1) a band-pass filter with a frequency range of 0.5 - 40 Hz is used to remove power line interference and the DC component in the EEG; (2) independent component analysis (ICA) is used to remove blink artifacts in the EEG for subsequent feature extraction; (3) to enrich the data, partial overlap of the EEG data is allowed when setting the time window length. The EEG data during the viewing of emotional videos is overlapped, with a maximum overlap set to 1000 milliseconds, one EEG data segment every 2000 milliseconds, and a sampling frequency of 250 Hz.

[0057] For the DEAP dataset, the signals in DEAP are downsampled to 128 hz. In this application, 60 seconds of EEG data after the baseline (3s) is taken from the first 32 EEG channels, and the number of samples for each subject is 2400 (40 trials × 60 seconds). In this application, the labels for the 60 data segments of each trial are set to be the same, and the threshold is set to 5. When the valence or arousal value is greater than 5, the corresponding label is set to 1; when it is less than or equal to 5, the corresponding label is set to 0.

[0058] DE feature extraction is performed on the EEG data after artifact removal. For the DCEMD dataset of each subject, a 2s overlapping sliding window is used for data segmentation, and for the DEAP dataset, a 1s non-overlapping sliding window is used for data segmentation. For each data sample X i , the number of EEG data acquisition channels is 32; frequency domain features in five frequency bands of δ (1 - 3HZ), θ (4 - 7HZ), α (8 - 13HZ), β (14 - 30HZ), and γ (31 - 50HZ) are extracted.

[0059] S102: Use the adaptive graph learning method to learn the EEG data, and obtain feature maps of different scales by performing convolution and pooling operations at different levels, and construct a graph convolutional network for multi-scale feature reconstruction.

[0060] The main parameters in the neural network MMoGCN model include:

[0061] ① The expert network E, which includes two heterogeneous networks, MSFR-GCN and GCN, to extract task-shared and task-specific features respectively.

[0062] ② The number of tasks T.

[0063] ③ The parameter α of the dynamic weighted loss function i .

[0064] S103: Combine the graph convolutional network based on feature adaptive graph connection and the graph convolutional network for multi-scale feature reconstruction to construct a heterogeneous expert network.

[0065] This application uses two shared expert networks to extract shared features, namely, the GCN and MSFR-GCN integrated with a feature adaptive graph connection module. Specifically, for the input feature X ∈ R n×d (where n and d are the number of EEG channels and frequency bands respectively), this application adopts an adaptive graph learning method to form the feature adaptive graph connection of this application, dynamically learns the graph structure to construct the brain connection, and thus forms the adjacency matrix A. In the feature adaptive graph connection, the adjacency matrix A ij = g(x i , x j )(where i, j ∈ {1, 2, …, N}) represents the connection relationship between nodes x i and x j , and g(·) is defined as:

[0066]

[0067] where w is a learnable vector, and ReLU (Rectified Linear Unit) is a commonly used activation function, which is defined as:

[0068] ReLU(x) = max(0, x).

[0069] That is, when the input value x is greater than zero, the output is x; when the input value x is less than or equal to zero, the output is zero.

[0070] The above method subtracts x j from x i , and then normalizes it using the SoftMax operation to obtain the connection weight A ij . The shared expert network 1 directly uses the matrix A and the feature X for graph convolution, while the shared expert network 2 combines the matrix A with the reconstructed EEG feature X re , where X re uses multi-scale feature reconstruction to form the input of graph convolution. The two-layer graph convolution is applied as follows:

[0071]

[0072] S104: Use the constructed heterogeneous expert network to extract the task-shared features of the EEG data.

[0073] Among them, is the normalized Laplacian matrix; I is the n-order identity matrix; D is the diagonal matrix of A; W (1) and W (2) are learnable weight matrices. In order to pool the features into a fixed size, global pooling of the graph is applied. The pooled feature E can be expressed as:

[0074]

[0075] Among them, Z i is the feature vector of the i-th node in the output of the graph convolution operation; N is the number of nodes in the graph; finally, the pooled feature E is used as the final output of the expert network.

[0076] Furthermore, in an exemplary embodiment, step S2 can be replaced by the following steps.

[0077] S201: Based on the gating network, use the formula G (t) = softmax(W (t) x + b (t) ) to obtain the weight distribution of the task.

[0078] S202: Based on the weight distribution of the task, use the formula to obtain the expert weights W 0 , W 1 ; among them, G (t) is the weight distribution of task t, W (t) is the weight matrix of the gating network, x is the input feature representation, b (t) is the bias vector; O (t) is the weighted expert weight of task t; N is the number of experts; i is the corresponding expert number; is the weight distribution of expert network i generated by the gating network for task t; E i is the output of expert network i.

[0079] As Figure 3 shown, the gating network assigns different expert weights to each task, thus realizing dynamic task allocation. Each task has a corresponding gating network, which outputs a SoftMax-based weight distribution when receiving the input features, which determines the contribution of the outputs of different experts.

[0080] Furthermore, in an exemplary embodiment, step S3 can be replaced by the following steps.

[0081] S301: Input the expert weights into the tower layer module; the tower layer module is used to map the expert weights to the category space corresponding to the task and determine the initial classification result of the task.

[0082] As Figure 4 shown, in the model, the output of each task comes from the weighted combination of the experts selected by its gating network. Specifically, the weights generated by the gating network are used to perform weighted summation on the expert output E i . The specific structure of Tower is as Figure 4As shown, the module consists of three fully connected networks. Behind each layer, there are Batch Normalization (BatchNorm), LeakyReLU activation function, and Dropout to ensure the model's expressive ability and prevent overfitting. This setting gradually maps the features from the number of nodes to the number of classification categories. The role of the Tower module is to map the weighted expert output to a task-specific category space.

[0083] S302: Based on the initial classification results of the tasks, use the formula L (t) = crossentropy(1, 1 P ) + α||Θ|| 2 to obtain the classification results for each task; where crossentropy is the cross-entropy loss; 1 is the true label vector of the training data, and the training data is historical EEG data; 1 P is the predicted label vector of the training data; Θ is all model parameters; ||·|| 2 is the regularization weight, representing the l2 norm; L (t) is the task loss weight for each task; α is the trade-off regularization weight.

[0084] As Figure 5 shown, this application uses backpropagation to iteratively update the network parameters until the best result is obtained. Both emotion recognition and emotion disorder recognition are classification tasks. Therefore, this application uses the widely adopted cross-entropy loss function to calculate the classification loss. The cross-entropy loss quantifies the difference between the predicted label and the true label, and the regularization term α||Θ|| can prevent the overfitting of model parameters and limit the adjacency matrix.

[0085] In AWL, this application focuses on the homoscedasticity assumption, assuming that for a fixed input, the output variance between tasks remains unchanged. Therefore, AWL emphasizes modeling the uncertainty weights separately for each classification task and introduces a dynamic loss combination formula. The goal of AWL is to let the model automatically learn the weights of the task losses instead of setting them manually. These weights are learned through optimization and adjusted during backpropagation.

[0086] Furthermore, in an exemplary embodiment, step S4 can be replaced by the following steps.

[0087] S401: Based on the neural network model, use the task loss weight of each task to assign uncertainty weights to each task in the EEG data.

[0088] S402: Introduce a dynamic weighted loss function into the uncertainty weights; the dynamic loss function is

[0089] where L tis the loss of task t; σ t is the adaptive weight of task t, which is a trainable weight parameter; L AWL is the final weighted loss.

[0090] S403: Based on the dynamic weighted loss function, adjust the task loss weight according to the task after adjusting the uncertainty weight according to the assignment.

[0091] Further, in an exemplary embodiment, step S5 can be replaced by the following steps.

[0092] S501: Based on the adjusted task loss weight, use the Bayesian method to correct the bias of the neural network model to obtain a Bayesian model; the Bayesian model is an optimized neural network model.

[0093] In multi-task learning, the cross-task fusion loss is crucial. Traditionally, the loss weight of each task is manually adjusted to optimize the model performance. However, this method usually requires a large number of experiments and is impractical for determining the optimal weight in practical applications. To solve this problem, this application uses the Adaptive Weighted Loss (AWL) method to automatically balance the multi-task loss. The theory of AWL is rooted in uncertainty, modeled using the Bayesian method, and dynamically adjusts the loss weight of each task by modeling the task uncertainty, adaptively learning the weights to balance the learning difficulty and noise level between tasks. The uncertainty in the Bayesian model can be classified into the following categories:

[0094] Epistemic uncertainty: Derived from limited data, representing the limited learning ability of the model.

[0095] Aleatoric uncertainty: Caused by noise in the data, which can be further divided into: (1) data-dependent uncertainty (heteroscedasticity): Output uncertainty due to unstable input data quality. (2) task-dependent uncertainty (homoscedasticity): Inherent differences in learning ability between tasks, resulting in result uncertainty.

[0096] In AWL, this application focuses on the homoscedasticity assumption, assuming that for a fixed input, the output variance between tasks remains constant. Therefore, AWL emphasizes separately modeling the uncertainty weight for each classification task and introducing a dynamic loss combination formula. The goal of AWL is to let the model automatically learn the weights of the task losses instead of setting them manually. These weights are learned through optimization and adjusted during backpropagation.

[0097] This application specifically verifies the model performance on the DCEMD dataset and the DEAP dataset. The cross-subject dataset division on DCEMD is as Figure 6As shown, after the division is completed, the present application uses the leave-one-out cross-validation method to evaluate the model performance. The comparison of the final test results with the prior arts (MTL-EEGNet, MTL-ShallowConvNet, and MTL-DeepConvNet) is shown in Table 1:

[0098] Table 1 Comparison of classifier performance on the DCEMD dataset

[0099]

[0100]

[0101] The present application conducts a cross-subject experiment on the DEAP dataset to verify the performance of the model. The experiment uses the leave-one-out cross-validation method and evaluates two tasks: valence classification and arousal classification. The comparison of the final test results with the prior arts (such as MT-MKL, PLRSA, and TSception) is shown in Table 2:

[0102] Table 2 Comparison of classifier performance on the DEAP dataset

[0103] Classifier MT-MKL PLRSA TSception This application Potency accuracy 60.00 / - 61.84 / - 62.27 / - 64.52 / 7.3 Arousal accuracy 56.00 / - 62.07 / - 63.75 / - 67.39 / 9.7

[0104] From the above experimental results, it can be seen that the present application outperforms the prior arts on both the DCEMD dataset and the DEAP dataset, verifying the effectiveness and advancement of the proposed model. Specifically, on the DCEMD dataset, the model of the present application achieved accuracies of 50.69% / 5.0 and 77.88% / 14.2 in the emotion recognition and emotion disorder detection tasks respectively, significantly exceeding the prior methods such as MTL-EEGNet, MTL-ShallowConvNet, and MTL-DeepConvNet.

[0105] On the DEAP dataset, the present application achieved accuracies of 64.52% / 7.3 and 67.39% / 9.7 in the valence classification and arousal classification tasks respectively through the cross-subject experiment, showing a significant improvement compared with methods such as MT-MKL, PLRSA, and TSception. This further verifies the superior performance of the present application in the multi-task emotion recognition task.

[0106] Through comprehensive analysis, the proposed multi-task learning framework effectively integrates the shared features of the emotion and emotion disorder tasks, fully explores the potential connection between the two, and improves the classification performance and generalization ability of the model. In addition, the verification results of the model on the two datasets show that it has strong adaptability and robustness, providing an effective new method for emotion and emotion disorder recognition of EEG signals.

[0107] In an exemplary embodiment, the embodiment of the present application provides an EEG emotion recognition and emotion disorder detection device based on multi-task learning, specifically including:

[0108] A task-sharing feature extraction module, where a heterogeneous expert network extracts task-sharing features of EEG data.

[0109] A neural network model construction module, based on the task-sharing features, assigns corresponding expert weights to each task through a gating network, and constructs a neural network model with historical EEG data as the input and historical task classification results as the output.

[0110] A task loss weight determination module, inputs the EEG data after calculating the expert weights into the tower layer module of the neural network model for task classification to obtain the classification results of each task.

[0111] A task loss weight adjustment module, based on the neural network model, assigns uncertainty weights to each task in the EEG data, and dynamically adjusts the task loss weights according to the tasks after assigning uncertainty weights through a dynamic weighted loss function.

[0112] An optimized neural network model determination module, corrects the bias of the neural network model according to the adjusted task loss weights, reconstructs the neural network model, and determines the optimized neural network model; inputs the EEG data into the optimized neural network model to determine the task classification results.

[0113] In an exemplary embodiment, a computer device is provided. The computer device can be a server or a terminal, and its internal structure diagram can be as Figure 7 shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O), and a communication interface. Among them, the processor, the memory, and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video tag processing data. The input / output interface of the computer device is used to exchange information between the processor and external devices. The communication interface of the computer device is used to communicate with external terminals through a network connection. The computer program, when executed by the processor, implements a video tag processing method.

[0114] In an exemplary embodiment, a computer-readable storage medium is provided, storing a computer program which, when executed by a processor, implements the steps in the above-described embodiment of the EEG emotion recognition and emotion disorder detection method based on multi-task learning.

[0115] The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity of description, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, it should be considered as the scope recorded in this specification.

[0116] Specific examples are used in this article to elaborate on the principles and implementation manners of the present application. The description of the above embodiments is only used to help understand the method and its core idea of the present application; at the same time, for those of ordinary skill in the art, according to the idea of the present application, there will be changes in the specific implementation manners and application scopes. In summary, the content of this specification should not be construed as a limitation to the present application.

Claims

1. A method for EEG emotion recognition and emotional disorder detection based on multi-task learning, characterized in that: The EEG emotion recognition and emotional disorder detection method based on multi-task learning includes: Extracting task-shared features from EEG data based on heterogeneous expert networks; Based on the task sharing features, a corresponding expert weight is assigned to each task through a gated network, and a neural network model is constructed with historical EEG data as input and historical task classification results as output; Inputting the EEG data after calculating the expert weights into the tower layer module in the neural network model to perform task classification, and obtaining the classification results of each task; Based on the neural network model, an uncertainty weight is assigned to each task in the EEG data, and a task loss weight is dynamically adjusted according to the task after the uncertainty weight is assigned through a dynamic weighted loss function; According to the adjusted task loss weights, the bias of the neural network model is corrected, the neural network model is reconstructed, and an optimized neural network model is determined; The EEG data is input into the optimized neural network model to determine the task classification result; the task classification result includes emotional state detection and emotional disorder detection; the emotional state detection includes positive emotion, negative emotion and neutral emotion; the emotional disorder detection includes normal emotion and emotional disorder.

2. The EEG emotion recognition and emotional disorder detection method based on multi-task learning according to claim 1, characterized in that: The task-shared features of EEG data are extracted based on heterogeneous expert networks, including: Using an adaptive graph learning method to learn the EEG data, and constructing a graph convolutional network based on feature adaptive graph connections; Using an adaptive graph learning method to learn the EEG data, performing convolution and pooling operations at different levels to obtain feature maps of different scales, and constructing a graph convolution network for multi-scale feature reconstruction; Combining the graph convolutional network based on feature adaptive graph connection with the graph convolutional network for multi-scale feature reconstruction, a heterogeneous expert network is constructed; The constructed heterogeneous expert network is used to extract task-shared features of the EEG data.

3. The EEG emotion recognition and emotional disorder detection method based on multi-task learning according to claim 1, characterized in that: Based on the task sharing features, a corresponding expert weight is assigned to each task through a gated network, and a neural network model is constructed with historical EEG data as input and historical task classification results as output, specifically including: Based on the gated network, using formula G (t) =softmax(W (t) x+b (t) )Get the weight distribution of tasks; Based on the weight distribution of the tasks, using the formula Get the expert weight corresponding to the task; where G (t) is the weight distribution of task t, W (t) is the weight matrix of the gating network, x is the input feature representation, b (t) is the deviation vector; O (t) is the weighted expert weight of task t; N is the number of experts; i is the corresponding expert number; is the weight distribution of the expert network i generated by the gating network for task t; E i is the output of expert network i.

4. The EEG emotion recognition and emotional disorder detection method based on multi-task learning according to claim 1, characterized in that: The EEG data after calculating the expert weights is input into the tower layer module in the neural network model for task classification to obtain the classification results of each task, specifically including: Inputting the expert weights into a tower layer module; the tower layer module is used to map the expert weights to a category space corresponding to the task, and determine an initial classification result of the task; Based on the initial classification results of the task, using formula L (t) =crossentropy ( 1,1 P ) +α||Θ||2 to obtain the classification results of each task; where crossentropy is the cross entropy loss; 1 is the true label vector of the training data, and the training data is the historical EEG data; 1 P is the predicted label vector of the training data; Θ is all model parameters; ||·||2 is the regularization weight, representing the l2 norm; L (t) is the task loss weight for each task; α is the trade-off regularization weight.

5. The EEG emotion recognition and emotional disorder detection method based on multi-task learning according to claim 4, characterized in that: The tower layer module includes three layers of fully connected networks; Among them, in the first layer of the fully connected network, h1=Dropout(LeakyReLU(BatchNorm(W1O (t) +b1))), input feature O (t) Mapped to the output dimension of the first layer of the fully connected network, O (t) is the input of the first layer of fully connected network; W1 is the weight matrix of the first layer of fully connected network; b1 is the bias vector of the first layer of fully connected network; h1 is the output of the first layer of fully connected network; In the second-layer fully connected network, h2=Dropout(LeakyReLU(BatchNorm(W2h1+b2))) is used to map h1 to the dimension-reduced space, and obtain the output h2 of the second-layer fully connected network; where W2 is the weight matrix of the second layer; b2 is the bias vector of the second-layer fully connected network; In the third layer of the fully connected network, using y (t) = Dropout(LeakyReLU(BatchNorm(W3h2+b3))), maps h2 to the task-specific class number to obtain the predicted category of task t; the predicted category is the initial classification result of the task; where W3 is the weight matrix of the third-layer fully connected network; b3 is the bias vector of the third layer; y (t) is the predicted category of task t.

6. The EEG emotion recognition and emotional disorder detection method based on multi-task learning according to claim 1, characterized in that: Based on the neural network model, an uncertainty weight is assigned to each task in the EEG data, and a dynamic weighted loss function is used to dynamically adjust the task loss weight according to the task after the uncertainty weight is assigned, specifically including: Based on the neural network model, using the task loss weight of each task, assigning an uncertainty weight to each task in the EEG data; A dynamic weighted loss function is introduced into the uncertainty weight; the dynamic loss function is Among them, L t is the loss of task t; σt is the adaptive weight of task t, which is a trainable weight parameter; L AWL is the final weighted loss. Based on the dynamic weighted loss function, the task loss weights are adjusted according to the tasks assigned with the adjusted uncertainty weights.

7. The EEG emotion recognition and emotional disorder detection method based on multi-task learning according to claim 1, characterized in that: According to the adjusted task loss weight, the bias of the neural network model is corrected, the neural network model is reconstructed, and the optimized neural network model is determined, which specifically includes: Based on the adjusted task loss weights, the bias of the neural network model is corrected using the Bayesian method to obtain a Bayesian model; the Bayesian model is an optimized neural network model.

8. An EEG emotion recognition and emotional disorder detection device based on multi-task learning, characterized in that: The EEG emotion recognition and emotional disorder detection device based on multi-task learning includes: Task-shared feature extraction module: heterogeneous expert network extracts task-shared features of EEG data; A neural network model building module, based on the task sharing features, assigns corresponding expert weights to each task through a gated network, and uses historical EEG data as input and historical task classification results as output to build a neural network model; The task loss weight determination module inputs the EEG data after calculating the expert weight into the tower layer module in the neural network model to perform task classification and obtain the classification result of each task; A task loss weight adjustment module, which assigns uncertainty weights to each task in the EEG data based on the neural network model, and dynamically adjusts the task loss weights according to the tasks after the uncertainty weights are assigned through a dynamic weighted loss function; The optimized neural network model determination module corrects the bias of the neural network model according to the adjusted task loss weight, reconstructs the neural network model, and determines the optimized neural network model; inputs the EEG data into the optimized neural network model to determine the task classification result.

9. A computer device comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the EEG emotion recognition and emotional disorder detection method based on multi-task learning described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the EEG emotion recognition and emotional disorder detection method based on multi-task learning described in any one of claims 1 to 7 is implemented.

Images