Emotion recognition network pre-training method and device, equipment and medium

Abstract

The application is suitable for the technical field of software, and provides a pre-training method and device of an emotion recognition network, equipment and a medium, comprising: performing feature extraction on electroencephalogram data samples through a feature extractor to obtain an emotion feature vector; performing domain classification on the emotion feature vector through a domain discriminator, and calculating a domain classification loss according to a domain classification result and a domain label; performing emotion classification on the emotion feature vector through a classifier, and calculating an emotion classification loss according to an emotion classification result and a real label; performing OOD detection on the emotion feature vector through an OOD detector, and calculating an OOD detection loss according to an OOD detection result and the real label; performing enhancement type prediction on the emotion feature vector of the unlabeled sample through a first classification head, and calculating an enhancement classification loss of the unlabeled sample according to an enhancement prediction type of the unlabeled sample and a real enhancement type; updating parameters according to the calculated losses, continuing to train the network, and ending the pre-training until the pre-training ends.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of software, and particularly relates to a pre-training method and device of an emotion recognition network, an equipment and a storage medium. BACKGROUND

[0002] Emotion recognition of electroencephalogram (EEG) signals is one of the core research directions in the field of affective computing and brain-computer interface, which realizes the objective recognition of human psychological state by analyzing the features related to emotion in EEG signals. EEG signals can more truly reflect the individual's internal emotional state due to their difficult-to-fake characteristics, and have important application value in the fields of mental health monitoring, human-computer interaction, etc. At present, the emotion recognition technology based on EEG still faces several key challenges.

[0003] 1. Individual differences and feature extraction bottleneck

[0004] EEG signals have significant individual differences and are easily disturbed by non-stationary and non-Gaussian noise, making it difficult to build a general affective computing model. Existing technologies rely on single-modal feature extraction methods, such as focusing only on the temporal or spatial features of EEG signals, without fully integrating multi-dimensional spatio-temporal information, limiting the model's generalization ability across subjects. Although transfer learning is introduced to alleviate individual differences (such as aligning the feature distribution of the source domain and the target domain through domain adversarial network DANN), it relies on the independent and identically distributed assumption of the source domain and the target domain, and is sensitive to noisy labels, which is not stable in practical applications. In addition, in traditional methods, hand-designed features (such as power spectrum energy, differential entropy, etc.) or deep learning-based feature extraction models (such as CNN, LSTM) are difficult to dynamically capture the functional connectivity relationship between EEG channels, resulting in insufficient discriminability of emotion-related features.

[0005] 2. Label data scarcity and noisy label interference

[0006] Supervised learning models rely heavily on a large amount of high-quality labeled data, but the annotation of EEG data requires professional knowledge and human cost, making it difficult to obtain labeled samples. In addition, in emotion induction experiments, subjects may not be able to accurately feedback their emotional state, introducing label noise and further reducing the robustness of the model. Although pairwise learning and label distribution learning (such as RGNN model) are used to alleviate the impact of noisy labels, the performance of these methods is limited in scenarios with very few labeled data. Semi-supervised learning (SSL) can improve the model's generalization ability by combining a small amount of labeled data with a large amount of unlabeled data, but existing SSL methods usually assume that labeled and unlabeled data share the same class space, without considering the problem that unlabeled data may mix in Out-of-Distribution (OOD) samples in actual scenarios.

[0007] 3. Out-of-distribution sample challenge in open-set semi-supervised learning

[0008] In practical applications, the unlabelled electroencephalogram data may contain OOD samples (such as baseline signals without induced emotions or signals in unknown emotional states) that do not belong to the labelled categories, and these outliers will interfere with model training, resulting in a decrease in recognition accuracy. Although existing research attempts to filter OOD samples through a multi-task framework (such as joint OOD detection and classification), there is a conflict between the OOD detection task and the In-Distribution (ID) classification task: the former needs to cluster ID samples into the same category, while the latter needs to enhance the class distinction between ID samples. This contradiction will impair feature learning and increase training difficulty. In addition, existing OOD detection methods usually require a large amount of ID labelled data, making it difficult to apply in low labelled resource scenarios.

[0009] Therefore, there is an urgent need for an emotion recognition method that can integrate multi-dimensional electroencephalogram features, adapt to individual differences, and effectively suppress the interference of OOD samples in an open-set semi-supervised scenario, in order to improve the accuracy and reliability of the model in real environments. SUMMARY

[0010] The purpose of the present application is to provide a pre-training method, device and equipment for an emotion recognition network, and a storage medium, aiming to solve the problem that the pre-training method of the existing emotion recognition network cannot effectively suppress the interference of OOD samples, resulting in poor accuracy and reliability of the network in real environments.

[0011] In a first aspect, the present application provides a pre-training method for an emotion recognition network, which pre-trains the emotion recognition network through a pre-training network, the pre-training network comprising a feature extractor, a classifier, a domain discriminator, an OOD detector and a first classification head, the emotion recognition network comprising the feature extractor and the classifier, and the pre-training method comprising the following steps:

[0012] extracting emotional features from the input electroencephalogram data samples in the electroencephalogram data sample group through the feature extractor to obtain an emotional feature vector, the electroencephalogram data sample group comprising labelled source domain electroencephalogram data samples, unlabelled source domain electroencephalogram data samples and unlabelled target domain electroencephalogram data samples;

[0013] inputting the emotional feature vector into the classifier, classifying the emotional feature vector through the classifier to obtain an emotional classification result, and calculating an emotional classification loss of the labelled sample according to the emotional classification result and the emotional label of the labelled sample;

[0014] input the emotion feature vector into the domain discriminator through a gradient inversion layer, perform domain classification on the emotion feature vector through the domain discriminator to obtain a domain classification result, and calculate a domain classification loss according to the domain classification result and a domain label;

[0015] input the emotion feature vector and the corresponding emotion classification result into the OOD detector, perform OOD detection on the emotion feature vector through the OOD detector to obtain an OOD detection result, and calculate an OOD detection loss of the electroencephalogram data sample of the labeled sample according to the OOD detection result of the labeled sample and an emotion label;

[0016] perform enhanced type prediction on the emotion feature vector of the unlabeled sample through the first classification head to obtain an enhanced predicted type of the unlabeled sample, and calculate an enhanced classification loss of the unlabeled sample according to the enhanced predicted type of the unlabeled sample;

[0017] update parameters of the pre-training network according to the emotion classification loss, the domain classification loss, the OOD detection loss and the enhanced classification loss, continue to train the emotion recognition network until pre-training is completed, and obtain a pre-trained emotion recognition network.

[0018] In some embodiments, before the step of performing emotion feature extraction on the input electroencephalogram data samples of the electroencephalogram data sample group through the feature extractor, the method comprises:

[0019] obtaining electroencephalogram data samples for training, if the obtained electroencephalogram data samples are unlabeled samples, respectively applying data enhancement of different amplitudes to the obtained electroencephalogram data samples to obtain data-enhanced electroencephalogram data samples, setting the data-enhanced electroencephalogram data samples as the electroencephalogram data samples of the electroencephalogram data sample group and inputting the data-enhanced electroencephalogram data samples to the feature extractor.

[0020] In some embodiments, the step of respectively applying data enhancement of different amplitudes to the obtained electroencephalogram data samples to obtain data-enhanced electroencephalogram data samples comprises:

[0021] applying three different amplitudes of additive Gaussian noise to the obtained electroencephalogram data samples to respectively generate non-enhanced electroencephalogram data samples, weakly enhanced electroencephalogram data samples and strongly enhanced electroencephalogram data samples.

[0022] In some embodiments, the step of performing OOD detection on the emotion feature vector through the OOD detector to obtain an OOD detection result comprises:

[0023] if the electroencephalogram data sample is a labeled sample, calculating a similarity score of the sample label based on the emotion feature vector of the electroencephalogram data sample and an emotion label.

[0024] If the electroencephalogram data sample is an unlabeled sample, a similarity score of a sample predicted label is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted label.

[0025] Otsu threshold segmentation is performed on the similarity scores of all samples, and samples lower than the Otsu threshold are marked as OOD samples or noise samples.

[0026] In some embodiments, the step of calculating a similarity score of a sample label based on the emotion feature vector of the electroencephalogram data sample and the emotion label comprises:

[0027] A first similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the emotion label, a second similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted hard label, and a third similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted soft label.

[0028] And / or, the step of calculating a similarity score of a sample predicted label based on the emotion feature vector of the electroencephalogram data sample and the predicted label comprises:

[0029] A fourth similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted hard label, and a fifth similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted soft label.

[0030] In a second aspect, the present application provides a fine-tuning method of an emotion recognition network, which fine-tunes the emotion recognition network by a fine-tuning network, the emotion recognition network being pre-trained by the pre-training method of any one of the above, the fine-tuning network comprising a pre-training network pre-trained by the pre-training method of any one of the above and a second classification head, and the fine-tuning method comprising the following steps:

[0031] An emotion feature extractor pre-trained by the pre-training method is used to extract emotion features from input electroencephalogram data samples in a sample group, to obtain a first emotion feature vector, the sample group comprising labeled source domain electroencephalogram data samples and unlabeled source domain electroencephalogram data samples.

[0032] The first emotion feature vector and the corresponding emotion classification result are input into an OOD detector pre-trained by the pre-training method, the OOD detector pre-trained by the pre-training method is used to detect the first emotion feature vector, to obtain an OOD detection result, and an OOD detection loss of an electroencephalogram data sample of an unlabeled sample is calculated based on the OOD detection result and the predicted label of the unlabeled sample.

[0033] If the OOD detection result is an ID sample, the first emotion feature vector is subjected to OOD feature pruning to obtain a second emotion feature vector, the first and second emotion feature vectors are input into the second classification head, and first and second predicted features are obtained correspondingly, and a pruning loss of the electroencephalogram data sample is calculated according to the first and second predicted features.

[0034] According to the OOD detection loss, the pruning loss and a fine-tuning loss of the pre-trained network after pre-training, the parameters of the fine-tuning network are updated, the training of the emotion recognition network is continued until fine-tuning is completed, so as to obtain a fine-tuned emotion recognition network.

[0035] In some embodiments, the step of subjecting the first emotion feature vector to OOD feature pruning to obtain a second emotion feature vector comprises:

[0036] An ID-OOD sample pair is generated by matching an OOD sample with the same label to the electroencephalogram data sample in the OOD sample queue.

[0037] The ID-OOD sample pair is subjected to feature vector orthogonal decomposition to obtain the second emotion feature vector.

[0038] In a third aspect, the present application provides a pre-training device for an emotion recognition network, which pre-trains the emotion recognition network through a pre-training network, the pre-training network comprising a feature extractor, a classifier, a domain discriminator, an OOD detector and a first classification head, the emotion recognition network comprising the feature extractor and the classifier, and the pre-training device comprising:

[0039] A feature extraction unit is configured to extract emotion features from an input electroencephalogram data sample in a group of electroencephalogram data samples through the feature extractor to obtain an emotion feature vector, the group of electroencephalogram data samples comprising labeled source domain electroencephalogram data samples, unlabeled source domain electroencephalogram data samples and unlabeled target domain electroencephalogram data samples.

[0040] A first loss calculation unit is configured to input the emotion feature vector into the classifier, perform emotion classification on the emotion feature vector through the classifier to obtain an emotion classification result, and calculate an emotion classification loss of a labeled sample according to the emotion classification result and an emotion label of the labeled sample.

[0041] A second loss calculation unit is configured to input the emotion feature vector into the domain discriminator through a gradient reversal layer, perform domain classification on the emotion feature vector through the domain discriminator to obtain a domain classification result, and calculate a domain classification loss according to the domain classification result and a domain label.

[0042] a third loss calculation unit, configured to input the emotion feature vector and the corresponding emotion classification result into the OOD detector, perform OOD detection on the emotion feature vector through the OOD detector to obtain an OOD detection result, and calculate an OOD detection loss of the EEG data sample of the labeled sample according to the OOD detection result and the emotion label of the labeled sample;

[0043] a fourth loss calculation unit, configured to perform enhanced type prediction on the emotion feature vector of the unlabeled sample through the first classification head to obtain an enhanced prediction type of the unlabeled sample, and calculate an enhanced classification loss of the unlabeled sample according to the enhanced prediction type of the unlabeled sample;

[0044] a parameter updating unit, configured to update the parameters of the pre-training network according to the emotion classification loss, the domain classification loss, the OOD detection loss and the enhanced classification loss, continue to train the emotion recognition network until pre-training is completed, and obtain a pre-trained emotion recognition network.

[0045] In a fourth aspect, the present application also provides a computing device, which comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, and the processor implements the steps of the above method when executing the computer program.

[0046] In a fifth aspect, the present application also provides a computer readable storage medium, which stores a computer program, and the computer program implements the steps of the above method when executed by a processor.

[0047] In the pre-training of the emotion recognition network, the emotion feature vector and the corresponding emotion classification result are input into the OOD detector, the OOD detector performs OOD detection on the emotion feature vector to obtain an OOD detection result, the OOD detection loss of the EEG data sample of the labeled sample is calculated according to the OOD detection result and the emotion label of the labeled sample, the emotion feature vector of the unlabeled sample is predicted by the first classification head to obtain an enhanced prediction type of the unlabeled sample, the enhanced classification loss of the unlabeled sample is calculated according to the enhanced prediction type and the real enhanced type, and the parameters of the pre-training network are updated based on the OOD detection loss and the enhanced classification loss, so that the feature space is shared with the classifier during OOD detection, the feature conflict is avoided, the noise label and the abnormal sample are identified, the training of the emotion recognition network is effectively prevented, the unlabeled sample is self-supervised trained through data enhancement identification, the generalization ability of the feature extractor is improved, and the robustness of the pre-trained emotion recognition network is better. BRIEF DESCRIPTION OF DRAWINGS

[0048] Figure 1 is a flowchart of a pre-training method of an emotion recognition network provided by an embodiment of the present application;

[0049] Figure 2 is a flowchart of an implementation process of OOD detection on the emotion feature vector by the OOD detector to obtain an OOD detection result provided by an embodiment of the present application;

[0050] Figure 3 is a structural diagram of a pre-training network of an emotion recognition network provided by an embodiment of the present application;

[0051] Figure 4 is a flowchart of a fine-tuning method of an emotion recognition network provided by an embodiment of the present application;

[0052] Figure 5 is a structural diagram of a fine-tuning network of an emotion recognition network provided by an embodiment of the present application;

[0053] Figure 6 is a structural diagram of a pre-training device of an emotion recognition network provided by an embodiment of the present application;

[0054] Figure 7 is a structural diagram of a computing device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0055] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0056] It should be understood that when used in the specification and the appended claims, the term "comprising" indicates the presence of the described features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. And the terms used in the specification of the present application are only for the purpose of describing specific embodiments and are not intended to limit the present application. As used in the specification and the appended claims of the present application, the singular forms "a", "an" and "the" are intended to include the plural forms unless the context clearly indicates otherwise. The terms "first", "second" and similar terms do not indicate any order, number or importance, but are only used to distinguish different components. The terms "connected" or "connected" and the like do not limit to physical or mechanical connections, but can include electrical connections, whether direct or indirect. "Up", "down", "left", "right", and the like are only used to indicate relative positional relationships, which may change accordingly when the absolute positions of the described objects change.

[0057] For the sake of clarity and conciseness of the following description of embodiments of the present application, detailed description of some known functions and known components is omitted.

[0058] The specific implementation of the present application is described in detail below in combination with specific embodiments:

[0059] Figure 1 An implementation process of a pre-training method of an emotion recognition network provided by an embodiment of the present application is shown, only parts related to the embodiments of the present application are shown for the sake of convenience of description, and are described in detail as follows.

[0060] The embodiments of the present application are applicable to computing devices, for example, personal computers, servers, to pre-train an emotion recognition network on the computing devices by a pre-built pre-training network, the pre-training network including a feature extractor, a classifier, a domain discriminator, an OOD detector and a first classification head, wherein the emotion recognition network includes the feature extractor and the classifier in the pre-training network.

[0061] In step S101, the electroencephalogram data samples of the input electroencephalogram data sample group are subjected to emotion feature extraction by the feature extractor to obtain emotion feature vectors.

[0062] In the embodiments of the present application, the emotion recognition network is pre-trained using the electroencephalogram data samples of an electroencephalogram data sample group (Batch) each time, the electroencephalogram data sample group including labeled source domain electroencephalogram data samples, unlabeled source domain electroencephalogram data samples and unlabeled target domain electroencephalogram data samples, the source domain samples including the labeled source domain electroencephalogram data samples and the unlabeled source domain electroencephalogram data samples, so as to better adapt to the scenario of label scarcity, and the unlabeled target domain electroencephalogram data samples can train the generalization and robustness of the network across datasets. As an example, the labeled source domain electroencephalogram data samples and the unlabeled target domain electroencephalogram data samples can contain three emotion categories: positive, neutral and negative, and the unlabeled source domain electroencephalogram data samples can contain four emotion categories: positive, neutral, negative and fear, wherein "fear" can be regarded as an OOD category.

[0063] The electroencephalogram data samples can be electroencephalogram (EEG), and the feature extractor is used to extract features from the electroencephalogram data samples to obtain emotion feature vectors. Specifically, the feature extractor can be a multi-layer fully connected network to extract electroencephalogram time sequence features, thereby providing a high-dimensional semantic feature space for subsequent classification and domain alignment, for example, the multi-layer fully connected network can be a three-layer fully connected network with a structure of 310-64-64-32.

[0064] In step S102, the emotion feature vector is input into the classifier, the emotion classification of the emotion feature vector is performed through the classifier, an emotion classification result is obtained, and an emotion classification loss of the labeled sample is calculated according to the emotion classification result and the emotion label of the labeled sample.

[0065] In the embodiment of the present application, the classifier is used to perform emotion classification on the emotion feature vector to obtain an emotion classification result, that is, an emotion prediction vector or an emotion probability distribution. Specifically, the classifier can include a fully connected layer and a Softmax layer, the fully connected layer is used to map the high-dimensional semantic feature to an emotion category space for generating original classification prediction values (logits), and then the Softmax layer is used to perform Softmax conversion on the original classification prediction values to generate the emotion probability distribution.

[0066] In some embodiments, when the emotion classification loss of the labeled sample is calculated according to the emotion classification result and the emotion label of the labeled sample, a cross-entropy loss is calculated using the emotion classification result and the emotion label of the labeled sample to update the parameters of the classifier and the feature extractor at the same time through back propagation, so that the network gradually improves the emotion classification capability. Specifically, the emotion classification loss of the labeled sample is calculated using the formula , wherein n represents the number of samples in the electroencephalogram data sample group, represents the emotion classification result (emotion prediction vector) of the labeled sample, represents the emotion label of the labeled sample.

[0067] In step S103, the emotion feature vector is input into the domain discriminator through the gradient reversal layer, the domain classification of the emotion feature vector is performed through the domain discriminator to obtain a domain classification result, and a domain classification loss is calculated according to the domain classification result and the domain label.

[0068] In the embodiment of the present application, the gradient reversal layer does not change the feature in the forward propagation, so that the emotion feature vector is input into the domain discriminator through the gradient reversal layer (GRL), the domain classification (binary classification) of the emotion feature vector is performed through the domain discriminator to obtain the domain classification result, that is, the source domain or the target domain, and when the parameters of the feature extractor are updated thereafter, the gradient of the domain classification loss is reversed, so that the feature extractor is updated in the direction of “domain indistinguishable”.

[0069] In some embodiments, when the domain classification loss is calculated according to the domain classification result and the domain label, a binary cross-entropy loss is calculated according to the domain classification result and the domain label to measure the difficulty of domain distinction, and the loss is back propagated (after being reversed by GRL) to the feature extractor to realize the learning of the domain invariant feature. Specifically, the domain classification loss is calculated using the following formula: , wherein Indicates the parameters of the feature extractor. The parameters of the domain discriminator, Indicates a field label, Indicates the domain classification result, , Representation domain discriminator, This represents the emotion feature vector. During backpropagation, it is... Introducing a gradient inversion layer in the optimization While discriminating against the discriminator, the feature extractor is forced to learn domain-invariant representations, achieving adversarial training. In this way, the feature extractor maximizes the classification ability, while the discriminator... This minimizes the domain differences, reducing the distributional discrepancies between the source and target domains. The objective function during domain adversarial training can be expressed as:

[0070]

[0071] The goal is to maximize the domain classification loss and minimize the emotion classification loss during training. This represents the emotion classification loss for labeled samples (source domain). Representation domain classification loss, This represents hyperparameters used to ensure the stability of domain adversarial operations. , where p is a parameter related to the training round, which is the ratio of the current training round to the maximum training round.

[0072] In step S104, the emotion feature vector and the corresponding emotion classification result are input into the OOD detector. The OOD detector performs OOD detection on the emotion feature vector to obtain the OOD detection result. Based on the OOD detection result of the labeled sample and the emotion label, the OOD detection loss of the labeled EEG data sample is calculated.

[0073] In this embodiment of the invention, the emotion feature vector of the EEG data sample and the corresponding emotion classification result are input into an OOD detector. The OOD detector performs OOD detection on the emotion feature vector to obtain the OOD detection result, thereby identifying noise-labeled samples and OOD samples and preventing them from interfering with emotion classification. Specifically, the emotion feature vector of the EEG data sample and the emotion classification result (vector) can be concatenated, and the concatenated vector is input into the OOD detector, which can be a fully connected neural network.

[0074] Figure 2 The following is an optional implementation of step S104 when performing OOD detection on the emotion feature vector using an OOD detector to obtain the OOD detection result:

[0075] In step S201, if the electroencephalogram data sample is a labeled sample, a similarity score of the sample label is calculated based on the emotion feature vector of the electroencephalogram data sample and the emotion label.

[0076] In step S202, if the electroencephalogram data sample is an unlabeled sample, a similarity score of the sample predicted label is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted label.

[0077] In step S203, Otsu threshold segmentation is performed on the similarity scores of all samples, and samples below the Otsu threshold are marked as OOD samples or noise samples.

[0078] In the embodiment of the present application, OOD sample or noise sample detection is performed based on the matching degree of the electroencephalogram data sample and the label, to prevent interference with the classification of the classifier.

[0079] In some embodiments, if the electroencephalogram data sample is a labeled sample, a similarity score of the sample label is calculated based on the emotion feature vector of the electroencephalogram data sample and the emotion label, specifically including the following steps:

[0080] A first similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the emotion label, a second similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted hard label, and a third similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted soft label.

[0081] In the embodiment of the present application, the predicted hard label of the electroencephalogram data sample is a label different from the emotion label (true label) but with the highest prediction probability, and the predicted soft label is a randomly selected class prediction label after excluding the true label and the predicted hard label.

[0082] In some embodiments, if the electroencephalogram data sample is an unlabeled sample, a similarity score of the sample predicted label is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted label, specifically including the following steps:

[0083] A fourth similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted hard label, and a fifth similarity score is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted soft label.

[0084] In the embodiment of the present application, the predicted hard label of the electroencephalogram data sample is a label with the highest prediction probability, and the predicted soft label is a randomly selected class prediction label after excluding the predicted hard label.

[0085] In the embodiment of the present application, considering that the pre-training network is not stable when pre-training the emotion recognition network, therefore, the parameters of the pre-training network are not updated using the OOD detection result of the labeled sample. In this way, after obtaining the OOD detection result, only the OOD detection loss of the labeled sample is calculated according to the OOD detection result of the labeled sample and the emotion label, so as to update the parameters of the OOD detector and continuously improve its detection capability. Specifically, the OOD detection loss of the labeled sample is calculated by the following formula: wherein, represents the similarity score of the emotion feature vector and the emotion label represents the similarity score of the emotion feature vector and the predicted hard label represents the similarity score of the emotion feature vector and the predicted soft label In this way, the OOD detection loss of the labeled sample guides the parameter update of the OOD detector more accurately.

[0086] In step S105, the emotion feature vector of the unlabeled sample is enhanced type prediction by the first classification head, and the enhanced prediction type of the unlabeled sample is obtained. According to the enhanced prediction type of the unlabeled sample, the enhanced classification loss of the unlabeled sample is calculated.

[0087] In the embodiment of the present application, in order to make full use of the OOD sample to effectively train the emotion recognition network and enhance the representation ability of the feature extractor to all training samples, including ID and OOD samples, while not impairing the classification performance of the emotion recognition network, the unlabeled sample is used for self-supervised pre-training of the emotion recognition network. Specifically, the emotion feature vector of the unlabeled sample output by the feature extractor is enhanced type prediction by the first classification head, and the enhanced prediction type of the unlabeled sample is obtained. Then, according to the enhanced prediction type of the unlabeled sample, the enhanced classification loss of the unlabeled sample is calculated to guide the subsequent parameter update of the feature extractor.

[0088] In some embodiments, the first classification head can be a multi-classifier to perform enhanced type prediction on the emotion feature vector.

[0089] ​​In some embodiments, in order to perform data augmentation on the unlabeled samples, therefore, before performing emotion feature extraction on the input EEG data samples of the EEG data sample group by the feature extractor, the EEG data samples for training are obtained, if the obtained EEG data samples are unlabeled samples, different amplitudes of data augmentation are respectively applied to the obtained EEG data samples to obtain the data-augmented EEG data samples, the data-augmented EEG data samples are set as the EEG data samples of the EEG data sample group and input to the feature extractor. Specifically, when different amplitudes of data augmentation are respectively applied to the obtained EEG data samples, three different amplitudes of additive Gaussian noise are applied to the obtained EEG data samples to respectively generate non-augmented EEG data samples, weakly augmented EEG data samples and strongly augmented EEG data samples. At this time, the first classification head is a three-classifier, and the augmented classification loss of the unlabeled sample can be calculated by the following formula: wherein, represents the augmented prediction type of the unlabeled sample.

[0090] In step S106, the parameters of the pre-training network are updated according to the domain classification loss, the emotion classification loss, the OOD detection loss and the augmented classification loss, the training of the emotion recognition network is continued, and the pre-training is continued until the pre-training is ended to obtain the pre-trained emotion recognition network.

[0091] In the embodiments of the present application, after obtaining the domain classification loss, the emotion classification loss, the OOD detection loss and the augmented classification loss, the parameters of the pre-training network are updated, the training of the emotion recognition network is continued, and the pre-training is ended when the validation set classification accuracy or the total loss based on the domain classification loss, the emotion classification loss, the OOD detection loss and the augmented classification loss converges, thereby obtaining the pre-trained emotion recognition network, and also obtaining the pre-trained pre-training network.

[0092] The embodiment of the present application establishes a stable and highly generalizable emotion feature vector through a large number of labeled source domain electroencephalogram data samples, unlabeled source domain electroencephalogram data samples and unlabeled target domain electroencephalogram data samples for training. When pre-training the emotion recognition network, the emotion feature vector and the corresponding emotion classification result are input into the OOD detector. The OOD detector performs OOD detection on the emotion feature vector to obtain an OOD detection result. According to the OOD detection result of the labeled sample and the emotion label, the OOD detection loss of the electroencephalogram data sample of the labeled sample is calculated. The first classification head is used to predict the enhanced type of the emotion feature vector of the unlabeled sample to obtain the enhanced prediction type of the unlabeled sample. According to the enhanced prediction type and the real enhanced type of the unlabeled sample, the enhanced classification loss of the unlabeled sample is calculated. The parameters of the pre-training network are updated based on the OOD detection loss and the enhanced classification loss, thereby reducing the influence of individual differences and noise labels, enhancing the robustness of the emotion recognition network to EEG data, and enabling the emotion recognition network to have a better initialization state. Thus, when the subsequent emotion recognition network is fine-tuned, the learned general features can be used for further optimization, which can significantly improve the efficiency and final performance of the overall training of the emotion recognition network.

[0093] By way of example, Figure 3 The pre-training network of the emotion recognition network of the embodiment of the present application is shown, which includes a feature extractor, a classifier, a domain discriminator, an OOD detector and a first classification head. The emotion recognition network includes the feature extractor and the classifier in the pre-training network.

[0094] In the embodiment of the present application, after the pre-training network receives the input unlabeled source domain electroencephalogram data sample and the unlabeled target domain electroencephalogram data sample, different amplitudes of data enhancement are applied to the obtained electroencephalogram data samples respectively to obtain data-enhanced electroencephalogram data samples. The data-enhanced electroencephalogram data samples are set as the electroencephalogram data samples of the electroencephalogram data sample group and input to the feature extractor, while the labeled source domain electroencephalogram data samples are directly input to the feature extractor. The feature extractor extracts emotion features from the input electroencephalogram data samples of the electroencephalogram data sample group to obtain emotion feature vectors. Then, the classifier classifies the emotion feature vectors to obtain emotion classification results, the domain discriminator classifies the emotion feature vectors to obtain domain classification results, the first classification head predicts the enhanced type of the emotion feature vectors of the unlabeled samples to obtain the enhanced prediction type of the unlabeled samples, and the OOD detector performs OOD detection on the emotion feature vectors to obtain OOD detection results. Finally, the parameters of the pre-training network are updated according to the loss of the pre-training network, and the emotion recognition network is continuously trained until the pre-training is completed to obtain the pre-trained emotion recognition network.

[0095] Figure 4 An implementation process of the emotion recognition network fine-tuning method provided by the embodiment of the present application is shown, only the part related to the embodiment of the present application is shown for the convenience of description, and the details are described as follows.

[0096] The embodiment of the present application is applicable to a computing device, for example, a personal computer, a server, to fine-tune the pre-trained emotion recognition network on the computing device by the pre-constructed fine-tuning network, and finally obtain a trained emotion recognition network, the fine-tuning network includes the pre-trained network pre-trained by any of the foregoing pre-training methods and the second classification head, that is, the fine-tuning network includes the second classification head, the pre-trained feature extractor, the classifier, the domain discriminator, the OOD detector and the first classification head. When the pre-trained emotion recognition network is fine-tuned by the fine-tuning network, the pre-trained pre-training network and the second classification head continue to train together.

[0097] In step S401, the EEG data samples of the input EEG data sample set are subjected to emotion feature extraction by the pre-trained feature extractor to obtain first emotion feature vectors.

[0098] In the embodiment of the present application, the EEG data sample set includes labeled source domain EEG data samples and unlabeled source domain EEG data samples, and the pre-trained emotion recognition network is continuously trained by the labeled source domain EEG data samples and the unlabeled source domain EEG data samples to fine-tune the emotion recognition network.

[0099] In step S402, the first emotion feature vectors and the corresponding emotion classification results are input into the pre-trained OOD detector, the first emotion feature vectors are subjected to OOD detection by the pre-trained OOD detector to obtain OOD detection results, and the OOD detection loss of the EEG data samples of the unlabeled samples is calculated according to the OOD detection results of the unlabeled samples and the predicted labels.

[0100] In the embodiment of the present application, the first emotion feature vectors are subjected to emotion classification by the pre-trained classifier to obtain emotion classification results, the first emotion feature vectors and the corresponding emotion classification results are input into the pre-trained OOD detector, the first emotion feature vectors are subjected to OOD detection by the pre-trained OOD detector to obtain OOD detection results, and the OOD detection loss of the EEG data samples of the unlabeled samples is calculated according to the OOD detection results of the unlabeled samples and the predicted labels, to realize open set semi-supervised learning, wherein the unlabeled samples are the unlabeled source domain EEG data samples.

[0101] In some embodiments, the following loss function can be used to calculate the OOD detection loss of the EEG data samples of the unlabeled samples:

[0102] wherein, denotes the emotion feature vector a similarity score with a predicted hard label , the predicted hard label being the class label with the highest probability value, and the predicted soft label being a randomly selected class label after removing the predicted hard label. denotes the emotion feature vector a similarity score with a predicted soft label , the predicted hard label being the class label with the highest probability value, and the predicted soft label being a randomly selected class label after removing the predicted hard label.

[0103] In step S403, if the OOD detection result is an ID sample, OOD feature pruning is performed on the first emotion feature vector to obtain a second emotion feature vector, the first and second emotion feature vectors are input into the second classification head, and the first and second predicted features are correspondingly obtained, and the pruning loss of the electroencephalogram data sample is calculated according to the first and second predicted features.

[0104] In the embodiment of the present application, if the OOD detection result is an ID sample, OOD feature pruning is performed on the first emotion feature vector to obtain an ID feature representation after OOD semantics is removed (a second emotion feature vector), and finally the first and second emotion feature vectors are input into the second classification head, the first and second predicted features are correspondingly obtained, and the pruning loss of the electroencephalogram data sample is calculated according to the first and second predicted features. The pruning loss is used for subsequent consistency constraint to maintain consistent prediction direction and ensure stable convergence of the emotion recognition network.

[0105] In step S404, the parameters of the fine-tuning network are updated according to the OOD detection loss, the pruning loss, and the fine-tuning loss of the pre-trained pre-training network, and the emotion recognition network is continuously trained until the fine-tuning is completed to obtain the fine-tuned emotion recognition network.

[0106] In the embodiment of the present application, when the fine-tuning network is used to fine-tune the pre-trained emotion recognition network, the pre-trained pre-training network and the second classification head are also continuously trained together. Therefore, when the pre-training network is fine-tuned, the parameters of the fine-tuning network are updated according to the fine-tuning loss of the pre-trained pre-training network, the OOD detection loss calculated in step S402, and the pruning loss, the emotion recognition network is continuously trained until the fine-tuning is completed to obtain the fine-tuned emotion recognition network, that is, the trained emotion recognition network.

[0107] In the embodiment of the present application, the individual differences of samples and label pollution can be alleviated through noise detection and feature pruning. In addition, by pruning the useless semantic information in the OOD samples through OOD feature pruning on the emotion feature vector corresponding to the ID samples, the robustness of the emotion recognition network to the ID features is enhanced. Then, in the open set scene, the emotion recognition network is semi-supervised trained by combining OOD detection and self-supervised enhancement, realizing high-robust learning of the emotion recognition network in the case of label scarcity.

[0108] In some embodiments, when the first emotion feature vector of the ID electroencephalogram data sample is pruned by OOD feature, an ID-OOD sample pair is generated by matching an OOD sample with the same label in the OOD sample queue for the ID electroencephalogram data sample, the ID-OOD sample pair is subjected to orthogonal decomposition of the feature vector, and the second emotion feature vector is obtained. Thus, the feature is pruned by orthogonal decomposition, the useless semantic information in the OOD sample is removed, and the robustness of the emotion recognition network to the ID feature is enhanced, and the stability of the classification boundary is strengthened.

[0109] Specifically, the first emotion feature vector of the electroencephalogram data sample matches an OOD sample feature vector with the same label in the feature vector queue of the OOD sample, an ID-OOD sample feature pair is generated, and the ID-OOD sample feature pair is subjected to orthogonal decomposition of the feature vector to obtain a second emotion feature vector. When the ID-OOD sample feature pair is subjected to orthogonal decomposition of the feature vector, the projection of the OOD sample feature in the direction of the ID sample feature is calculated, and the pruned ID sample feature is obtained based on the soft orthogonal decomposition. In the embodiment of the present application, the projection vector represents the semantic component in the OOD sample feature that overlaps with the ID sample feature direction. By subtracting a part of the calculated projection vector from the original ID sample feature vector, the pruned ID sample feature vector can be obtained, thereby obtaining the ID sample feature representation after removing part of the OOD overlapping semantics.

[0110] In the embodiment of the present application, the pruning loss of the electroencephalogram data sample can be calculated by the following formula:

[0111] and

[0112] ;

[0113] wherein, and represent the orthogonal decomposition loss of the labeled samples and the unlabeled samples, respectively, C represents the category or label number, K represents the number of labels, represents the labeled ID sample in the electroencephalogram data sample group, represents the labeled ID sample, denote the first predicted feature and the second predicted feature of the unlabeled sample, denote the sample label, M denote the total number of the unlabeled samples, denote the unlabeled ID sample, denote the unlabeled ID sample in the EEG data sample set, denote the first predicted feature and the second predicted feature of the unlabeled sample.

[0114] In the embodiment of the present application, for the unlabeled sample, the cropping loss of the EEG data sample is calculated by the KL divergence, and for the labeled sample, the cross-entropy between the second predicted feature and the sample label is minimized, so that the cross-entropy loss is additionally utilized to stabilize the feature space structure by utilizing the label information, thereby improving the generalization performance of the emotion recognition network.

[0115] As an example, Figure 5 a fine-tuning network of the emotion recognition network in the embodiment of the present application is shown, which includes the pre-training network pre-trained by any of the foregoing pre-training methods and the second classification head, that is, the fine-tuning network includes the second classification head, the pre-trained feature extractor, the classifier, the domain discriminator, the OOD detector, and the first classification head. When the pre-trained emotion recognition network is fine-tuned by the fine-tuning network, the pre-trained pre-training network and the second classification head continue to be trained together.

[0116] Figure 6 The structure of a pre-training device of an emotion recognition network provided in an embodiment of the present application is shown, and the emotion recognition network is pre-trained by a pre-training network, the pre-training network includes a feature extractor, a classifier, a domain discriminator, an OOD detector, and a first classification head, the emotion recognition network includes the feature extractor and the classifier in the pre-training network, only parts related to the embodiments of the present application are shown for convenience of description, and the pre-training device includes:

[0117] a feature extraction unit 61, configured to perform emotion feature extraction on EEG data samples in an input EEG data sample set by a feature extractor to obtain an emotion feature vector, the EEG data sample set including labeled source domain EEG data samples, unlabeled source domain EEG data samples, and unlabeled target domain EEG data samples;

[0118] The first loss calculation unit 62 is configured to input the emotion feature vector into a classifier, perform emotion classification on the emotion feature vector by using the classifier to obtain an emotion classification result, and calculate an emotion classification loss of the labeled sample according to the emotion classification result of the labeled sample and an emotion label;

[0119] The second loss calculation unit 63 is configured to input the emotion feature vector into a domain discriminator through a gradient inversion layer, perform domain classification on the emotion feature vector by using the domain discriminator to obtain a domain classification result, and calculate a domain classification loss according to the domain classification result and a domain label;

[0120] The third loss calculation unit 64 is configured to input the emotion feature vector and the corresponding emotion classification result into an OOD detector, perform OOD detection on the emotion feature vector by using the OOD detector to obtain an OOD detection result, and calculate an OOD detection loss of the labeled sample of the electroencephalogram data sample according to the OOD detection result of the labeled sample and the emotion label.

[0121] The fourth loss calculation unit 65 is configured to perform enhanced type prediction on the emotion feature vector of the unlabeled sample by using a first classification head to obtain an enhanced prediction type of the unlabeled sample, and calculate an enhanced classification loss of the unlabeled sample according to the enhanced prediction type of the unlabeled sample.

[0122] The parameter updating unit 66 is configured to update parameters of the pre-training network according to the emotion classification loss, the domain classification loss, the OOD detection loss and the enhanced classification loss, continue to train the emotion recognition network until pre-training is completed, and obtain the pre-trained emotion recognition network.

[0123] In the embodiments of the present application, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified, and in actual application, the above-mentioned functions can be allocated by different functional units and modules, that is, the internal structure of the device is divided into different functional units or modules to realize all or part of the functions described above. Each unit and module of the device can be realized by a corresponding hardware or software unit, and each unit and module can be an independent software and hardware unit, or can be integrated into a software and hardware unit, which is not limited herein. In addition, the specific names of each functional unit and module are only for easy distinction, and do not limit the protection scope of the present application. The specific working process of the units and modules in the device can be referred to the corresponding description in the foregoing pre-training method embodiments, which will not be repeated here.

[0124] Figure 7 The structure of the computing device provided by an embodiment of the present application is shown, and only the parts related to the embodiments of the present application are shown for the convenience of description.

[0125] The computing device 7 of the embodiment of the present application comprises a processor 70, a memory 71, and a computer program 72 stored in the memory 71 and executable on the processor 70. The processor 70 implements the steps in each of the above method embodiments when executing the computer program 72, for example Figure 1 the steps S101-S106 shown. Alternatively, the processor 70 implements the functions of each unit in the above device embodiments when executing the computer program 72, for example Figure 6 the functions of the units 61-66 shown.

[0126] The computing device of the embodiment of the present application can be a personal computer, a server. The steps implemented by the processor 70 in the computing device 7 when executing the computer program 72 to implement the above method can refer to the description of the foregoing method embodiments, which will not be described here.

[0127] In an embodiment of the present application, a computer readable storage medium is also provided, which stores a computer program. The computer program is executed by a processor to implement the steps in the above method embodiments, for example, Figure 1 the steps S101-S106 shown. Alternatively, the computer program is executed by a processor to implement the functions of each unit in the above device embodiments, for example Figure 6 the functions of the units 61-66 shown.

[0128] In the pre-training of the emotion recognition network, the embodiment of the present application shares the feature space with the classifier during OOD detection, avoids feature conflict, and can effectively prevent interference with the training of the emotion recognition network by identifying noise labels and abnormal samples. Through data enhancement identification, self-supervised training is performed on unlabeled samples to improve the generalization ability of the feature extractor, so that the robustness of the pre-trained emotion recognition network is better. In the fine-tuning of the pre-trained emotion recognition network, noise detection and feature pruning can alleviate sample individual differences and label pollution. In addition, by pruning the useless semantic information in the OOD sample through OOD feature pruning of the emotion feature vector corresponding to the ID sample, the robustness of the emotion recognition network to ID features is enhanced, and then the emotion recognition network is semi-supervised trained in the open set scene by combining OOD detection and self-supervised enhancement, realizing high-robust learning of the emotion recognition network in the case of label scarcity.

[0129] The computer readable storage medium of embodiments of the present application may, for example, be but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the computer readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In embodiments of the present application, the computer readable storage medium can be any tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0130] The above embodiments are merely intended for describing the technical solutions of the present application, not to limit the present application. Although the present application is described in detail with reference to the above embodiments, those skilled in the art should understand that the disclosed scope of the above embodiments is not limited to the technical solutions formed by the specific combination of the technical features, and should also cover other technical solutions formed by any combination of the above technical features or equivalent features without departing from the disclosed concept. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

[0131] Further, although operations are depicted in a particular order, this should not be understood as requiring the operations to be performed in the particular order shown or in sequential order, and that certain operations can be performed in parallel or in any order. Also, although specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present application. Certain features described in the context of separate embodiments can also be implemented in combination. Conversely, various features described in the context of a single embodiment can also be implemented separately or in any suitable subcombination.

Claims

1. A pre-training method of an emotion recognition network, characterized in that, The emotion recognition network is pre-trained by a pre-training network, the pre-training network comprises a feature extractor, a classifier, a domain discriminator, an OOD detector and a first classification head, the emotion recognition network comprises the feature extractor and the classifier, and the pre-training method comprises the following steps: emotion feature extraction is performed on the EEG data samples in the input EEG data sample group by the feature extractor to obtain emotion feature vectors, the EEG data sample group comprises labeled source domain EEG data samples, unlabeled source domain EEG data samples and unlabeled target domain EEG data samples; the emotion feature vectors are input into the classifier, emotion classification is performed on the emotion feature vectors by the classifier to obtain emotion classification results, and emotion classification loss of the labeled samples is calculated according to the emotion classification results and emotion labels of the labeled samples; the emotion feature vectors are input into the domain discriminator through a gradient reversal layer, domain classification is performed on the emotion feature vectors by the domain discriminator to obtain domain classification results, and domain classification loss is calculated according to the domain classification results and domain labels; the emotion feature vectors and corresponding emotion classification results are input into the OOD detector, OOD detection is performed on the emotion feature vectors by the OOD detector to obtain OOD detection results, and OOD detection loss of the EEG data samples of the labeled samples is calculated according to the OOD detection results and emotion labels of the labeled samples; enhanced type prediction is performed on the emotion feature vectors of the unlabeled samples by the first classification head to obtain enhanced prediction types of the unlabeled samples, and enhanced classification loss of the unlabeled samples is calculated according to the enhanced prediction types of the unlabeled samples; the parameters of the pre-training network are updated according to the emotion classification loss, the domain classification loss, the OOD detection loss and the enhanced classification loss, the emotion recognition network is continuously trained until the pre-training is completed, and a pre-trained emotion recognition network is obtained.

2. The method of claim 1, wherein, Before the step of performing emotion feature extraction on the EEG data samples in the input EEG data sample group by the feature extractor, the following steps are included: obtaining EEG data samples for training, if the obtained EEG data samples are unlabeled samples, applying different amplitudes of data enhancement to the obtained EEG data samples respectively to obtain data-enhanced EEG data samples, setting the data-enhanced EEG data samples as the EEG data samples of the EEG data sample group and inputting them into the feature extractor.

3. The method of claim 2, wherein, The step of applying different amplitudes of data enhancement to the obtained EEG data samples respectively to obtain data-enhanced EEG data samples comprises: applying three different amplitudes of additive Gaussian noise to the obtained EEG data samples to generate non-enhanced EEG data samples, weakly enhanced EEG data samples and strongly enhanced EEG data samples respectively.

4. The method of claim 1, wherein, The step of performing OOD detection on the emotion feature vectors by the OOD detector to obtain OOD detection results comprises: If the electroencephalogram data sample is a labeled sample, a similarity score of a sample label is calculated based on the emotion feature vector of the electroencephalogram data sample and the emotion label; If the electroencephalogram data sample is an unlabeled sample, a similarity score of a sample predicted label is calculated based on the emotion feature vector of the electroencephalogram data sample and the predicted label; Otsu threshold segmentation is performed on the similarity scores of all samples, and samples below the Otsu threshold are marked as OOD samples or noise samples.

5. The method of claim 4, wherein: the step of calculating a similarity score of a sample label based on the emotion feature vector of the electroencephalogram data sample and the emotion label comprises: calculating a first similarity score based on the emotion feature vector of the electroencephalogram data sample and the emotion label, calculating a second similarity score based on the emotion feature vector of the electroencephalogram data sample and the predicted hard label, and calculating a third similarity score based on the emotion feature vector of the electroencephalogram data sample and the predicted soft label; and / or, the step of calculating a similarity score of a sample predicted label based on the emotion feature vector of the electroencephalogram data sample and the predicted label comprises: calculating a fourth similarity score based on the emotion feature vector of the electroencephalogram data sample and the predicted hard label, and calculating a fifth similarity score based on the emotion feature vector of the electroencephalogram data sample and the predicted soft label.

6. A method of fine-tuning an emotion recognition network, the method comprising: The emotion recognition network is fine-tuned by a fine-tuning network, the emotion recognition network being pre-trained by the pre-training method of any one of claims 1 to 5, the fine-tuning network comprising a pre-training network pre-trained by the pre-training method of any one of claims 1 to 5 and a second classification head, and the fine-tuning method comprising the following steps: emotion feature extraction is performed on the electroencephalogram data samples in an input electroencephalogram data sample group by a pre-trained feature extractor to obtain first emotion feature vectors, the electroencephalogram data sample group comprising labeled source domain electroencephalogram data samples and unlabeled source domain electroencephalogram data samples; the first emotion feature vectors and corresponding emotion classification results are input into a pre-trained OOD detector, the first emotion feature vectors are subjected to OOD detection by the pre-trained OOD detector to obtain OOD detection results, and an OOD detection loss of the electroencephalogram data samples of the unlabeled samples is calculated based on the OOD detection results and the predicted labels of the unlabeled samples; if the OOD detection result is an ID sample, OOD feature pruning is performed on the first emotion feature vectors to obtain second emotion feature vectors, the first and second emotion feature vectors are input into the second classification head to obtain first and second predicted features, and a pruning loss of the electroencephalogram data samples is calculated based on the first and second predicted features; the parameters of the fine-tuning network are updated based on the OOD detection loss, the pruning loss, and the fine-tuning loss of the pre-trained pre-training network, the emotion recognition network is continuously trained until fine-tuning is completed, and a fine-tuned emotion recognition network is obtained.

7. The method of claim 6, wherein, The step of performing OOD feature pruning on the first emotion feature vector to obtain a second emotion feature vector comprises: Matching a same-labeled OOD sample for the EEG data sample in the OOD sample queue to generate an ID-OOD sample pair; Performing feature vector orthogonal decomposition on the ID-OOD sample pair to obtain the second emotion feature vector. 8.A pre-training device of an emotion recognition network, characterized in that, The pre-training device comprises: The feature extraction unit is configured to perform emotion feature extraction on an input EEG data sample in an EEG data sample group by using the feature extractor to obtain an emotion feature vector, wherein the EEG data sample group comprises labeled source domain EEG data samples, unlabeled source domain EEG data samples, and unlabeled target domain EEG data samples. The first loss calculation unit is configured to input the emotion feature vector into the classifier, perform emotion classification on the emotion feature vector by using the classifier to obtain an emotion classification result, and calculate an emotion classification loss of a labeled sample according to the emotion classification result and an emotion label of the labeled sample. The second loss calculation unit is configured to input the emotion feature vector into the domain discriminator through a gradient reversal layer, perform domain classification on the emotion feature vector by using the domain discriminator to obtain a domain classification result, and calculate a domain classification loss according to the domain classification result and a domain label. The third loss calculation unit is configured to input the emotion feature vector and the corresponding emotion classification result into the OOD detector, perform OOD detection on the emotion feature vector by using the OOD detector to obtain an OOD detection result, and calculate an OOD detection loss of a labeled sample according to the OOD detection result and an emotion label of the labeled sample. The fourth loss calculation unit is configured to perform enhancement type prediction on the emotion feature vector of an unlabeled sample by using the first classification head to obtain an enhancement prediction type of the unlabeled sample, and calculate an enhancement classification loss of the unlabeled sample according to the enhancement prediction type. The parameter updating unit is configured to update parameters of the pre-training network according to the emotion classification loss, the domain classification loss, the OOD detection loss, and the enhancement classification loss, continue to train the emotion recognition network, and stop until pre-training is completed to obtain a pre-trained emotion recognition network.

9. A computing device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein, The processor implements the steps of the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium storing a computer program, the computer program comprising instructions that, when executed by a computer, cause the computer to perform the method of any one of claims 1 to 9. The computer program implements the steps of the method according to any one of claims 1 to 7 when executed by the processor.

Images