A multi-stream electroencephalogram emotion recognition method of local guiding global learning

Abstract

The application discloses a multi-stream electroencephalogram emotion recognition method of local guiding global learning, comprising the following steps: inputting four brain area local stream EEG signals into corresponding local stream modules respectively, and inputting the signals into local self-encoding feature enhancement modules to obtain local stream reconstruction signals and local classification results; calculating confidence weight of local stream guiding global stream; and using a global self-encoding feature enhancement module, a local guiding global learning module and a global classifier global classification result as an electroencephalogram emotion recognition result. The application overcomes the defects of blind overall modeling of a model on global brain data or lack of dynamic interaction among multi-streams in the prior art, generates weight in real time by using the classification performance of local streams of each functional area, guides the global stream to pay more attention to key brain area features, and thus improves the accuracy and robustness of emotion recognition.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of electroencephalogram emotion recognition, and in particular to a multi-stream electroencephalogram emotion recognition method based on local-to-global learning. BACKGROUND

[0002] With the development of human-computer interaction technology and the increasing demand for mental health monitoring, emotion recognition has become an important research topic in the field of the intersection of neuroscience, psychology and computer science. Among them, electroencephalogram (EEG) has gradually become an important data source for emotion recognition due to its high temporal resolution, non-invasiveness and ability to reflect brain activity in real time.

[0003] In recent years, deep learning techniques, especially convolutional neural networks (CNN), graph convolutional networks (GCN) and Transformer models, have been widely used to learn effective features in EEG signals, improving the accuracy of emotion recognition. However, existing methods often ignore the important differences in different functional areas of the brain in emotion processing, and often model the whole brain EEG signal as a whole, which can easily introduce redundant information from each brain region and reduce the efficiency of using key region features. Some methods divide multiple brain regions and use attention mechanisms to adaptively learn the importance of brain regions, but such non-explicit modeling depends on the quality of model training, and cannot directly reflect whether the important brain regions are truly distinguished. Existing methods usually only perform feature fusion at the end of the network, and cannot dynamically allocate learning resources according to the actual classification performance of each region, so it is difficult to fully utilize the differentiated roles of different functional areas in emotion processing, thereby affecting the recognition accuracy and robustness. SUMMARY

[0004] The present application aims to overcome the shortcomings of the prior art and provide a multi-stream electroencephalogram emotion recognition method based on local-to-global learning.

[0005] The purpose of the present application is achieved by the following technical solutions:

[0006] In a first aspect of the present application, a multi-stream electroencephalogram emotion recognition method based on local-to-global learning is provided, comprising the following steps:

[0007] The local stream module includes a local auto-encoding feature enhancement module and a local classifier connected in sequence, and the local auto-encoding feature enhancement module serves as a feature extractor; the local stream module includes four, corresponding to the frontal lobe, temporal lobe, parietal lobe and occipital lobe respectively;

[0008] The regional flow EEG signals of the four brain regions corresponding to the positions of the frontal lobe, the temporal lobe, the parietal lobe and the occipital lobe are respectively input into the local self-encoding feature enhancement module of the corresponding local flow module; the local self-encoding feature enhancement module comprises two layers of one-dimensional convolution units Conv1d connected in series and one layer of one-dimensional convolution unit Conv1d, wherein the regional flow EEG signal is obtained after passing through two layers of one-dimensional convolution units Conv1d connected in series to obtain intermediate features F, and then the intermediate features F are reconstructed to the same dimension as the input after passing through one layer of one-dimensional convolution unit Conv1d to obtain the local flow reconstruction signal; the intermediate features F further pass through the classifier comprising two layers of FC units to obtain the local classification result Logits of the regional flow EEG signal;

[0009] Based on the classification results Logits of the four local flow modules, the prediction probability of each local flow module for each class is calculated, wherein the class with the maximum prediction probability in each local flow module is the most possible class of the corresponding local flow module; the prediction advantage entropy is calculated by using the prediction probability of each local flow module for each class and the total number of classes C, and the composite confidence of the prediction result of each local flow module is further calculated by using the prediction advantage entropy and the most possible class of the corresponding local flow module; the composite confidences of the four local flow modules are normalized to obtain the confidence weight of the local flow guiding global flow;

[0010] The global flow module comprises a global self-encoding feature enhancement module, a local guided global learning module and a global classifier connected in sequence; the global self-encoding feature enhancement module has the same structure as the local self-encoding feature enhancement module, the EEG signal of the whole brain is input into the global self-encoding feature enhancement module to output the global flow feature signal, and the global flow feature signal is split into global local features corresponding to the positions; the local guided global learning module comprises four modules corresponding to the local flow modules of the positions of the frontal lobe, the temporal lobe, the parietal lobe and the occipital lobe, respectively; each local guided global learning module first flattens the intermediate features F output by the corresponding local flow module, then inputs the flattened features into a linear layer to obtain a single numerical value, and then maps the single numerical value to the interval [0, 1] using a Sigmoid function to obtain a local weight; the local flow intermediate features F output by each local flow module and the global local features corresponding to the positions are weighted and added using the local weight, and then multiplied by the confidence weight of the local flow guiding global flow to obtain local brain region features; all local brain region features are spliced to obtain global features; the global features are input into the global classifier to obtain global classification results;

[0011] The global classification results output by the global classifier are taken as the electroencephalogram emotion recognition results.

[0012] Further, the local flow module and the global flow module are trained local flow modules and global flow modules.

[0013] Further, the training process of the local flow module and the global flow module comprises:

[0014] The local cross-entropy loss is calculated by using the local classification result and the real local label result, the global cross-entropy loss is calculated by using the global classification result and the real global label result, the local reconstruction loss is calculated by using the local flow EEG signals of the frontal lobe, the temporal lobe, the parietal lobe and the occipital lobe corresponding positions and the local flow reconstruction signals output by the local self-encoding feature enhancement module, and the global reconstruction loss is calculated by using the EEG signals of the whole brain and the global flow feature signals output by the global self-encoding feature enhancement module.

[0015] The local flow module and the global flow module are trained based on the local cross-entropy loss, the global cross-entropy loss, the local reconstruction loss and the global reconstruction loss.

[0016] Further, in the training process, after the local flow reconstruction signal passes through the classifier containing two FC units and the local classification result Logits of the brain region local flow EEG signal is obtained, the following is further included:

[0017] The local flow module is pre-trained, t rounds of training are performed based on N test samples, if the similarity of N classification results of adjacent two rounds reaches a stable threshold, the continuous stable times are counted, and when the continuous stable times reaches a stable threshold, it is considered that the local flow module output is stable, and the parameters of the local flow module are saved.

[0018] The beneficial effects of the present application are:

[0019] In an exemplary embodiment of the present application, a multi-flow electroencephalogram emotion recognition method based on local flow dynamic guidance of global flow is provided to overcome the defects of blind overall modeling of the model on the whole brain data or lack of dynamic interaction among multi-flows in the prior art, and the classification performance of each functional area local flow is used to generate weights in real time to guide the global flow to pay more attention to the key brain area features, thereby improving the accuracy and robustness of emotion recognition. The local guidance global learning module can generate weights in real time according to the classification ability of each functional area local flow, guide the global flow to focus on the high contribution area features, and suppress the low correlation noise. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A flow chart of a local guidance global learning multi-flow electroencephalogram emotion recognition method provided for an exemplary embodiment of the present application is shown in FIG. 1.

[0021] Figure 2 A multi-flow network architecture diagram of a local guidance global learning multi-flow electroencephalogram emotion recognition method provided for an exemplary embodiment of the present application is shown in FIG. 2.

[0022] Figure 3A structural schematic diagram of a self-encoding feature enhancement module provided for an exemplary embodiment of the present application is shown.

[0023] Figure 4 A structural schematic diagram of a local guidance global learning module provided for an exemplary embodiment of the present application is shown.

[0024] Figure 5 A local flow module pre-training schematic diagram provided for an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION

[0025] The technical solutions of the present application will be described clearly and completely below with reference to the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0026] In the description of the present application, it should be noted that the directions or positional relationships indicated by "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like are described based on the directions or positional relationships described in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, "first", "second" are only for the purpose of description, and cannot be understood as indicating or implying relative importance.

[0027] In the description of the present application, it should be noted that, unless otherwise explicitly specified and limited, "mounting", "connection" and "connection" should be understood in a broad sense, for example, it can be fixed connection, or detachable connection, or integrally connected; it can be mechanical connection, or electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, or the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0028] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as there is no conflict.

[0029] Reference is made to Figure 1 , Figure 1 A flow chart of a local guidance global learning multi-flow electroencephalogram emotion recognition method provided by an exemplary embodiment of the present application is shown. Through a multi-flow network architecture, the learning of the global flow is dynamically guided by the weights generated in real time by the local brain area flow, so as to improve the accuracy and generalization performance of emotion recognition.

[0030] Figure 2The multi-stream network architecture on which the method of the present exemplary embodiment is based is shown, including four local stream modules in the upper half and one global stream module in the lower half. First, according to the division of the brain into regions in anatomy, the EEG channels are divided into four regions, region 1 to region 4, corresponding to the frontal lobe, temporal lobe, parietal lobe, and occipital lobe of the cerebral cortex, respectively (the overall data is also divided into 4 parts); the data of each brain region corresponds to a local stream module, and the EEG signals of the corresponding brain region will be processed through the local stream module and the emotion classification will be completed. At the same time, the EEG signals of the whole brain are input into a global stream module for processing and classification, and the final output classification result is the output of the global stream module. In the present exemplary embodiment, during the multi-task training of the multi-stream, the training effect of the local stream module can guide the global stream module to determine the importance of the signals of each brain region. Specifically:

[0031] A "stream" is defined as the complete process of inputting data into a model to obtain a prediction result, and the multi-stream network architecture on which the method is based includes four local stream modules and one global stream module. The structure of each stream module includes a feature extractor and a classifier, where the feature extractor is an asymmetric structure based on an autoencoder, named AFRM. AFRM is a lightweight module designed specifically for EEG signal feature extraction based on the idea of an autoencoder. Its core goal is to use reconstruction error to drive the network to automatically "filter out" noise or irrelevant components in the original signal, while retaining and strengthening those features that are most valuable for subsequent emotion classification or pattern recognition.

[0032] As shown in Figure 3 , AFRM accepts an input tensor with a shape of [channel number, frequency band number], first passes through two layers of one-dimensional convolution Conv1d, where the convolution kernel size is 1, only used to capture the association information between EEG channels; after that, the intermediate feature F still maintains the dimension of [channel number, frequency band number], which is a refined, deep, and recognizable representation; finally, a one-dimensional convolution layer reconstructs the intermediate feature back to a signal form with the same dimension as the input. In the training in the preferred exemplary embodiment, a self-encoding reconstruction loss is added during training, so that the network continuously optimizes the convolution kernel to focus on stable and meaningful EEG patterns under the constraint of "not changing the dimension".

[0033] AFRM is a feature extraction module in the local stream and the global stream, which outputs feature signals (local stream reconstruction signals, global stream feature signals), which are then input into the classifier for classification. The structure of the classifier is two layers of FC units, and the output result of each stream is a prediction of the emotion category.

[0034] Correspondingly, in the present exemplary embodiment, the local stream module comprises a local auto-encoding feature enhancement module AFRM and a local classifier connected in sequence, the local auto-encoding feature enhancement module serving as a feature extractor; the local stream module comprises four, corresponding to the frontal lobe, temporal lobe, parietal lobe and occipital lobe respectively, namely a frontal lobe auto-encoding feature enhancement module AFRM1, a temporal lobe auto-encoding feature enhancement module AFRM2, a parietal lobe auto-encoding feature enhancement module AFRM3 and an occipital lobe auto-encoding feature enhancement module AFRM4.

[0035] The local stream EEG signals of the four brain regions corresponding to the frontal lobe, temporal lobe, parietal lobe and occipital lobe are respectively input to the local auto-encoding feature enhancement modules of the corresponding local stream modules; the local auto-encoding feature enhancement module comprises two layers of one-dimensional convolution units Conv1d connected in series and one layer of one-dimensional convolution unit Conv1d, wherein the brain region local stream EEG signal is subjected to two layers of one-dimensional convolution units Conv1d connected in series to obtain intermediate features , and then the intermediate features are reconstructed into local stream reconstruction signals of the same dimension as the input brain region local stream EEG signals; the intermediate features are subjected to a classifier comprising two layers of FC units to obtain local classification results Logits of the brain region local stream EEG signals.

[0036] , wherein the local stream module of the rth (r takes values 1-4, representing the frontal lobe, temporal lobe, parietal lobe and occipital lobe respectively) brain region outputs a logits , B is the sample batch size, and C is the number of classifications (i.e. the number of emotion classification categories).

[0037] Then, based on the classification results Logits of the four local stream modules, the prediction probability of each local stream module for each category is calculated:

[0038] ;

[0039] In the formula, represents the prediction probability of the jth category output by the local stream module of the rth brain region, ; k represents a temporary variable, . Wherein the category with the maximum prediction probability in each local stream module is the most likely category of the corresponding local stream module, .

[0040] Although the prediction of the local stream still has differences under the same value, this can be measured by entropy, so the prediction entropy is calculated using the prediction probability of each local stream module for each category and the total number of categories C, and has:

[0041] ;

[0042] and further calculate the composite confidence of the prediction result of each local stream module by using the prediction merit entropy and the most probable class of the corresponding local stream module .

[0043] In order to facilitate the comparison of the differences between different local streams, while preventing the extreme case of information unable to be propagated to the downstream of the model due to the low confidence of all local stream modules, the composite confidences of the four local stream modules are finally normalized to obtain the confidence weight of the local stream guiding the global stream:

[0044] .

[0045] In order to integrate the features of the local stream and guide the learning of the model on the global feature according to the performance of each brain region, a local-guided global learning module LGL block is designed, and the specific structure is as shown in Figure 4 The structure is located in the global stream module, integrates the enhanced features of the four local stream modules and the global stream module, calculates the weight of the local stream feature through the attention mechanism, and weights the global stream feature by using the adaptive weight and the confidence weight. Specifically:

[0046] The global stream module includes a global self-encoding feature enhancement module, a local-guided global learning module, and a global classifier connected in sequence; the global self-encoding feature enhancement module has the same structure as the local self-encoding feature enhancement module (i.e. two layers of one-dimensional convolution units Conv1d and one layer of one-dimensional convolution unit Conv1d connected in series, which will not be described here), and the EEG signal of the whole brain is input into the global self-encoding feature enhancement module to output the global stream feature signal , and the global stream feature signal is split into the global local feature .

[0047] The local-guided global learning module includes four, respectively corresponding to the local stream modules of the frontal lobe, temporal lobe, parietal lobe, and occipital lobe, each local-guided global learning module first flattens the local stream intermediate feature output by the corresponding local stream module , inputs it into a linear layer (Linear layer) to obtain a single numerical value, and then uses a Sigmoid function to map it to the interval [0, 1] to obtain the local weight :

[0048]

[0049] The local stream intermediate feature output by each local stream module and the global local feature at the corresponding position are input into the local-guided global learning module with local weights The weighted sum is added and multiplied by the confidence weight of the local stream to guide the global stream to obtain local brain region features; all local brain region features are spliced to obtain global features

[0050]

[0051] is a global stream feature signal the local feature after splitting, indicates that different brain region features are spliced into a global feature.

[0052] The global feature is input into the global classifier (two FC units) to obtain a global classification result; the global classification result obtained by the global classifier output is taken as the electroencephalogram emotion recognition result.

[0053] As can be seen from the above, in the example embodiment, a multi-stream electroencephalogram emotion recognition method based on local stream dynamic guidance of global stream is provided to overcome the defects of blind overall modeling of the model on the whole brain data or lack of dynamic interaction among multi-streams in the prior art. The weight is generated in real time by using the classification performance of each functional area local stream to guide the global stream to pay more attention to the key brain region features, thereby improving the accuracy and robustness of emotion recognition. The local guidance global learning module can generate weights in real time according to the classification ability of each functional area local stream to guide the global stream to focus on high-contribution region features and suppress low-correlation noise.

[0054] More preferably, in an example embodiment, the local stream module and the global stream module are trained local stream modules and global stream modules.

[0055] More preferably, in an example embodiment, the training process of the local stream module and the global stream module includes:

[0056] (1) Calculate the local cross-entropy loss using the local classification result and the real local label result, and calculate the global cross-entropy loss using the global classification result and the real global label result.

[0057] Specifically, in the example embodiment, the basic cross-entropy (Cross-Entropy) loss is used to calculate the classification loss, and the formula is as follows:

[0058]

[0059] where C is the number of classes, is the one-hot representation of the true label of the i-th sample, is the predicted probability of the model for the c-th class, and N represents the number of samples. For the local cross-entropy loss​ , is the prediction probability of the c-th class by the local flow model, i.e., the local classification result logits The real local label result corresponds to the one-hot representation of the real local label of the i-th sample. For the global cross-entropy loss , is the prediction probability of the c-th class by the global flow model, i.e., the global classification result, and the real global label result corresponds to the one-hot representation of the real global label of the i-th sample:

[0060]

[0061] (2) The local reconstruction loss is calculated by using the mean square error loss of the local flow EEG signals of the frontal lobe, temporal lobe, parietal lobe and occipital lobe corresponding positions and the local flow reconstruction signals output by the local auto-encoding feature enhancement module, and the global reconstruction loss is calculated by using the mean square error loss of the EEG signals of the whole brain and the global flow feature signals output by the global auto-encoding feature enhancement module.

[0062] Specifically, in the present exemplary embodiment, the reconstruction loss is calculated by using the mean square error loss (Mean Squared Error Loss):

[0063]

[0064] wherein is the original data input, is the reconstruction output of the model in the AFRM (including the AFRM of the local flow module and the AFRM of the global flow module), and n represents the number of samples. Therefore, the local reconstruction loss of the local flow module and the global reconstruction loss of the global flow module are added to calculate.

[0065]

[0066] (3) Finally, the local flow module and the global flow module are trained based on the local cross-entropy loss, the global cross-entropy loss, the local reconstruction loss and the global reconstruction loss:

[0067]

[0068] More preferably, in an exemplary embodiment, during the training process, after the local flow reconstruction signal passes through the classifier containing two layers of FC units to obtain the local classification result Logits of the local flow EEG signal of the brain region, it further includes:

[0069] Pre-training the local flow module, t rounds of training are performed based on N test samples, if the similarity of N classification results of adjacent two rounds reaches a stable threshold The number of consecutive stabilities is counted, and when the number of consecutive stabilities reaches a stability threshold, it is considered that the local flow module output is stable, and the parameters of the local flow module are saved.

[0070] Specifically, in the present exemplary embodiment, if the local flow module and the global flow module are directly randomly initialized, the local flow module performs poorly in the early stage of training, and the guidance to the global flow module is not in place, or even random. Based on such consideration, in the present exemplary embodiment, the local flow module is pre-trained using the same data set, hoping that in the early stage of network framework training, the local flow module can well guide the learning of the global flow module.

[0071] However, there is another problem here, the common EEG emotion recognition data set is small, and the prior art only divides it into a training set and a test set. If the best result of the test set is directly saved during the pre-training process, it is not reasonable, because the labels of the test set cannot be known during the intermediate pre-training process, otherwise it is suspected of leaking the labels in advance. Therefore, in the present exemplary embodiment, it is observed during the process of training the model that if the model has not converged or is affected by noise, the prediction change is usually large; once the model tends to converge, the difference between adjacent batch predictions should be small. Therefore, the concept of prediction label stability is proposed, and when the local flow prediction is stable within a fixed number of epochs, the local flow model is saved.

[0072] As shown in Figure 5 , in the tthround of prediction, there is a prediction result , N is the number of test set samples, and the sample order is fixed, then the change rate of adjacent two round prediction labels can be defined as:

[0073]

[0074] wherein, is an indicator function, which takes 1 when the condition in the parentheses is true, and 0 otherwise; for t = 1, . Next, a stability threshold is introduced , when :

[0075]

[0076] When reaches a preset stability round threshold K (e.g. K 3), it is determined that the model output is already "stable" enough, and the local flow model is saved at this time.

[0077] For the pre-training manner of the local stream module, the same manner as the global training is adopted, that is, the local cross-entropy loss is calculated by using the local classification result and the real local label result, and the local reconstruction loss is calculated by using the four brain region local stream EEG signals of the frontal lobe, temporal lobe, parietal lobe and occipital lobe corresponding positions and the local stream reconstruction signal output by the local self-encoding feature enhancement module:

[0078]

[0079] On the public data set SEED-IV, the method of the above exemplary embodiment achieves an average recognition accuracy of 86.35%, and on the SEED data set, an accuracy of 97.59% is achieved, both of which are high.

[0080] Obviously, the above embodiments are only examples for the sake of clarity, and are not limiting of the embodiments. For those skilled in the art, other different forms of changes or variations can be made on the basis of the above description. It is not necessary and impossible to exhaust all the embodiments. The obvious changes or variations derived therefrom are still within the protection scope of the present application.

Claims

1. A multi-stream electroencephalogram (EEG) emotion recognition method of locally guiding global learning, characterized in that: The method comprises the following steps: The local stream module comprises a local self-encoding feature enhancement module and a local classifier connected in sequence, and the local self-encoding feature enhancement module serves as a feature extractor; the local stream module comprises four, corresponding to the frontal lobe, temporal lobe, parietal lobe and occipital lobe respectively; The local stream EEG signals of the four brain regions corresponding to the frontal lobe, temporal lobe, parietal lobe and occipital lobe are input into the local self-encoding feature enhancement modules of the corresponding local stream modules respectively; the local self-encoding feature enhancement module comprises two layers of one-dimensional convolution units Conv1d connected in series and one layer of one-dimensional convolution unit Conv1d, wherein the brain region local stream EEG signal is subjected to two layers of one-dimensional convolution units Conv1d connected in series to obtain intermediate features F, and then subjected to one layer of one-dimensional convolution unit Conv1d to reconstruct the intermediate features F back to the local stream reconstruction signal of the same dimension as the input; the intermediate features F are further subjected to a classifier comprising two layers of FC units to obtain the local classification result Logits of the brain region local stream EEG signal; Based on the classification results Logits of the four local stream modules, the prediction probability of each class of each local stream module is calculated, wherein the class with the maximum prediction probability in each local stream module is the most possible class of the corresponding local stream module; the prediction merit entropy is calculated by using the prediction probability of each class of each local stream module and the total number of classes C, and the composite confidence of the prediction result of each local stream module is further calculated by using the prediction merit entropy and the most possible class of the corresponding local stream module; the composite confidences of the four local stream modules are normalized to obtain the confidence weight of the local stream guiding the global stream; The global stream module comprises a global self-encoding feature enhancement module, a local guided global learning module and a global classifier connected in sequence; the global self-encoding feature enhancement module has the same structure as the local self-encoding feature enhancement module, the EEG signal of the whole brain is input into the global self-encoding feature enhancement module to output the global stream feature signal, and the global stream feature signal is split into global local features corresponding to the positions; the local guided global learning module comprises four, corresponding to the local stream modules of the frontal lobe, temporal lobe, parietal lobe and occipital lobe respectively, each local guided global learning module firstly flattens the intermediate features F output by the corresponding local stream module, then inputs into a linear layer to obtain a single numerical value, and then uses a Sigmoid function to map to the interval [0, 1] to obtain a local weight; the local stream intermediate features F output by each local stream module and the global local features corresponding to the positions are weighted and added by using the local weight, and then multiplied by the confidence weight of the local stream guiding the global stream to obtain local brain region features; all the local brain region features are spliced to obtain global features; the global features are input into the global classifier to obtain a global classification result; The global classification result output by the global classifier is taken as the electroencephalogram emotion recognition result; The calculation of the prediction merit entropy by using the prediction probability of each class of each local stream module and the total number of classes C comprises: ； In the formula, For predicting the superior and inferior entropy, is the prediction probability of each local flow module for each class, r represents the rth brain region, and j represents the jth class.

2. The method of claim 1, wherein the local-to-global learning guided multi-stream EEG emotion recognition method is characterized by: The local stream module and the global stream module are trained local stream modules and global stream modules.

3. The method of claim 2, wherein the local-to-global learning guided multi-stream EEG emotion recognition method is characterized by: The training process of the local flow module and the global flow module comprises: calculating a local cross-entropy loss by using the local classification result and the real local label result, calculating a global cross-entropy loss by using the global classification result and the real global label result, calculating a local reconstruction loss by using the four brain region local flow EEG signals corresponding to the frontal lobe, the temporal lobe, the parietal lobe and the occipital lobe and the local flow reconstruction signals output by the local self-encoding feature enhancement module, and calculating a global reconstruction loss by using the EEG signals of the whole brain and the global flow feature signals output by the global self-encoding feature enhancement module; training the local flow module and the global flow module based on the local cross-entropy loss, the global cross-entropy loss, the local reconstruction loss and the global reconstruction loss.

4. The method of claim 3, wherein the local-to-global learning guided multi-stream EEG emotion recognition method is characterized by: In the training process, after the local flow reconstruction signal passes through the classifier comprising two layers of FC units and obtains the local classification result Logits of the brain region local flow EEG signal, the following is further included: The local flow module is pre-trained, t rounds of training are performed based on N test samples, if the similarity of N classification results of adjacent two rounds reaches a stable threshold, the number of continuous stable times is counted, and when the number of continuous stable times reaches a stable threshold, it is considered that the local flow module output is stable, and the parameters of the local flow module are saved.

Images