Incremental learning strategy-based cross-individual electroencephalogram signal emotion recognition method

Abstract

The invention discloses a cross-individual electroencephalogram signal emotion recognition method based on an incremental learning strategy, and the method comprises the steps: firstly, importing collected electroencephalogram signal data, and carrying out the preprocessing and data enhancement of the data; secondly, dividing the data set, and cutting the data set into an original training set, an incremental training set and a test set; then, training a physiology-emotion recognition model (hereinafter referred to as TSN-LWF for short); and finally, based on the TSN-LWF model trained in the above steps, inputting and outputting test data to obtain a titer and wake-up dimension dichotomy or multi-classification task cross-individual electroencephalogram signal emotion recognition result.

Description

Technical Field

[0001] The present invention relates to the field of electroencephalogram (EEG) signal processing and pattern recognition, and in particular to a method for cross-individual EEG signal emotion recognition based on an incremental learning strategy. Background Art

[0002] Emotions are fundamental factors in human daily life, affecting decision-making, perception, interpersonal communication, and human intelligence. Emotion recognition plays an important role in cognitive behavioral therapy (CBT), emotion regulation therapy (ERT) / emotion-focused therapy (EFT), and the evaluation of drug treatments for emotion-related mental disorders such as generalized anxiety disorder (GAD) and depression. With the potential application of artificial intelligence (AI) in CBT and EFT, enabling artificial intelligence (AI) to recognize human emotions has attracted more and more researchers' interest in recent years.

[0003] Electroencephalography (EEG) is a widely used brain imaging technique that directly measures human brain activity. Several electrodes are placed on the surface of the human head to collect EEG signals. EEG has a high temporal resolution, so it can capture different brain states at the sub-second level. Brain-computer interface (BCI) systems can recognize human emotions through EEG with the help of machine learning and signal processing techniques.

[0004] Deep learning is a machine learning method based on artificial neural networks. By simulating the way the human brain processes information, it can automatically learn features and patterns from large amounts of data for a variety of complex tasks such as image recognition, speech processing, and natural language processing. Convolutional neural networks (CNNs) have shown promising results in BCI due to their ability to learn directly from EEG. Today, scholars have proposed deep and shallow convolutional neural networks, called DCN and SCN, which have achieved good results. Despite the advantages of CNN, due to the non-stationary and dynamic nature of EEG, a single-sized temporal kernel and shallow neural networks cannot effectively capture the neural processing of emotions that occur at different time scales and durations; at the same time, the variability of EEG signals between subjects limits its practical use and the versatility of classifiers, especially in subject-independent applications; in addition, the excessive number of EEG signal channels may result in excessive dimensions, excessive data volume, and a large amount of computing time. Summary of the invention

[0005] In view of the above problems, the object of the present invention is to solve part of the problems in the prior art, or at least alleviate these problems.

[0006] The present invention is achieved by at least one of the following technical solutions.

[0007] A method for cross-individual EEG signal emotion recognition based on an incremental learning strategy comprises the following steps:

[0008] Step 1: import the collected EEG data and perform preprocessing and data enhancement on the data, wherein the preprocessing includes artifact removal, filtering and EEG channel selection, and the data enhancement includes sample segmentation and time-frequency transformation;

[0009] Step 2: Divide the dataset into the original training set, incremental training set and test set. The original training set is used for basic training, the incremental training set is used for incremental training, and the test set is used to evaluate the effect of the physiological-emotion recognition model.

[0010] Step 3: Train the physiological-emotion recognition model (hereinafter referred to as TSN-LWF). The TSN-LWF model mainly consists of three modules, namely two parallel backbone networks (TSN1 and TSN2), two parallel output branches (N1 and N2) and an LWF-based classifier module; the training is divided into two stages. In the basic stage, the original training set is fed into the TSN-LWF model, the parameters of TSN1 and N1 are trained and the pre-trained model after convergence is saved; in the subsequent incremental stage, the output module N2 is randomly initialized, the parameters of the TSN1 module are migrated to TSN2, the TSN1 and N1 modules are fixed, the incremental training set is input into the model, and only the TSN2 and N2 modules are trained; the LWF-based classifier is to calculate the loss and optimize the model hyperparameters, and the model is trained using a public dataset;

[0011] Step 4: Output the results. Based on the TSN-LWF model trained in step 3, input the test set and output the emotion recognition results of cross-individual EEG signals in the binary or multi-classification tasks of valence and arousal dimensions. In the binary or multi-classification tasks, select 5 or the corresponding value as the threshold to divide each dimension into high / low levels.

[0012] Furthermore, the preprocessing includes artifact removal, filtering and EEG channel selection, specifically including:

[0013] First, the x-second pre-trial baseline of the EEG signal was deleted and the data was downsampled to 128 Hz; then, the independent component analysis method was used to remove the electrooculogram (EOG) mixed in the EEG signal; finally, in order to remove low-frequency and high-frequency noise interference, the EEG signal was filtered with a 4-45 Hz bandpass filter;

[0014] Existing physiological research results show that the frontal area stores more emotional information and is most closely related to emotions. The FP1-FP2 electrode pair and the F3-F4 electrode pair in the frontal area are located forward, in the area of ​​the forehead without hair obstruction, which is more conducive to placing dry electrodes to collect EEG signals. Therefore, the four EEG channels with the highest correlation with emotion recognition in the frontal area, namely FP1, FP2, F3, and F4, are selected.

[0015] The data enhancement includes sample segmentation and time-frequency transformation, as follows:

[0016] The EEG signal of the preprocessed data set is n seconds; a 1-second sliding window is used to cut the EEG signal in the time domain to significantly increase the number of samples;

[0017] For the input signal after sample segmentation, short-time Fourier transform is used to transform it into the frequency domain to extract features. The short-time Fourier transform formula is as follows:

[0018]

[0019] in is a window function, is the frequency, is a time index.

[0020] Furthermore, the data set is divided into the following parts:

[0021] The dataset is divided into the original training set, incremental training set and test set. The original training set is obtained by mixing the data of multiple individuals and has a large enough data volume for pre-training. 5% of the data of the new individual is divided into an incremental training set for process incremental training, and the remaining 95% of the data of the new individual constitutes the test set to evaluate the model effect.

[0022] Furthermore, in step 3, two parallel backbone networks (TSN1 and TSN2) are respectively connected to two parallel output modules (N1 and N2), the structures of TSN1 and TSN2 are the same, and the structures of N1 and N2 are the same; TSN1 and TSN2 are composed of a temporal convolution block, a spatial convolution block and a classifier module, N1 and N2 are both fully connected layers, and the output feature is m; the temporal convolution block is used to fully extract different frequency domain information, the spatial convolution block obtains a global feature representation by performing a convolution operation on the dimension of the EEG channel, and the classifier module is used to better obtain the emotion classification result.

[0023] Furthermore, the temporal convolution block is composed of three parallel multi-scale temporal convolutional layers. Each layer first performs a convolution operation, and the sizes of the convolution kernels are (1, 0.5*f), (1, 0.25*f), and (1, 0.15*f), respectively, where f refers to the number of sampling points. After the convolution, the RELU activation function and the average pooling layer are passed, and then the time-frequency features of different scales are connected along the feature dimension;

[0024] The spatial convolution block consists of two parallel multi-scale spatial convolution layers. Each layer first performs a convolution operation with kernel sizes of (0.5*c, 1) and (1*c, 1), where c refers to the number of electrodes. After the convolution, the RELU activation function, average pooling layer and batch normalization are used, and then spatial features of different scales are connected along the feature dimension.

[0025] The classifier module includes 3 ordinary convolution layers, 1 depthwise separable convolution layer and 2 connection layers; the input vector first passes through 3 ordinary convolution layers with the same convolution kernel size, and then passes through the depthwise separable convolution layer and obtains the output vector through the fully connected layer; the convolution kernel size of the ordinary convolution layer is 1×3, the step size is 1, the padding is 1, and the number of output channels is 32, 64, and 64 respectively; the depthwise separable convolution layer is composed of depthwise convolution and pointwise convolution, the depthwise convolution uses a 1x3 convolution kernel, and the pointwise convolution uses an output channel number of 128; after each convolution operation and depthwise separable convolution operation of the classifier module, batch normalization, RELU activation function, and maximum pooling are used to avoid model overfitting; the connection layer includes 2 fully connected layers with sizes of 512 and 128 respectively.

[0026] Furthermore, in step 3, the LWF-based classifier module is as follows:

[0027] The LWF (Less Forgetting Learning) method calls the trained TSN1 and N1 models the teacher models, and TSN2 and N2 the student models. In the basic stage, the teacher model is trained. In the subsequent incremental stage, in order to allow the student model to learn from the teacher model and obtain higher recognition accuracy when learning new data, the samples in the incremental training set must first pass through two parallel backbone networks and two parallel output modules. After prediction, a regular loss is defined. and one by conventional loss and distillation losses The combined loss function , and update the model using the corresponding loss function at each stage.

[0028] Furthermore, the combined loss function is specifically as follows:

[0029] Combined loss function By conventional loss and distillation losses composition,

[0030]

[0031] in is a network model, is the set of input data samples, is the corresponding true label, is the student model output, is the teacher model output, is the relaxation coefficient, is the set of parameters from the teacher model, are the student model parameters;

[0032] Regular losses To measure the degree to which a sample is correctly classified, a common multinomial logistic loss is used:

[0033]

[0034] in, is the corresponding true label, is the student model output,

[0035] Distillation loss Use KL divergence (Kullback-Leibler divergence, KLD for short) to evaluate the approximation between the two sets of probability distributions. By optimizing the KL divergence, the student model output is close to the recorded output of the teacher model, which will preserve the knowledge and transfer it to the new model.

[0036]

[0037] in and Represent the output of the teacher model and the student model respectively, is the distillation temperature;

[0038] In the basic stage, choose As the loss function, the parameters that need to be updated are the parameters in TSN1 and N1 , in the incremental phase, use As the loss function, the parameters of the output module N2 are randomly initialized , migrate TSN1 module parameters To TSN2, fix the parameters of TSN1 and N1 , and only update the parameters in TSN2 and N2, that is, .

[0039] It should be understood that the contents described in the summary of the invention are not intended to limit the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is the overall flow chart of the present invention;

[0041] Figure 2 It is the overall framework diagram of the TSN-LWF model of the present invention;

[0042] Figure 3 It is the overall framework diagram of the backbone network of the present invention; DETAILED DESCRIPTION

[0043] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0044] like Figure 1 , Figure 2 As shown, this embodiment provides a method for cross-individual EEG signal emotion recognition based on an incremental learning strategy, and the method includes the following steps:

[0045] Step 1: import the collected EEG data and perform preprocessing and data enhancement on the data, wherein the preprocessing includes artifact removal, filtering and EEG channel selection, and the data enhancement includes sample segmentation and time-frequency transformation;

[0046] The preprocessing includes artifact removal, filtering and EEG channel selection, specifically including:

[0047] First, the x-second pre-trial baseline of the EEG signal was deleted and the data was downsampled to 128 Hz; then, the independent component analysis method was used to remove the electrooculogram (EOG) mixed in the EEG signal; finally, in order to remove low-frequency and high-frequency noise interference, the EEG signal was filtered with a 4-45 Hz bandpass filter;

[0048] Existing physiological research results show that the frontal area stores more emotional information and is most closely related to emotions. The FP1-FP2 electrode pair and the F3-F4 electrode pair in the frontal area are located forward, in the area of ​​the forehead without hair obstruction, which is more conducive to placing dry electrodes to collect EEG signals. Therefore, the four EEG channels with the highest correlation with emotion recognition in the frontal area, namely FP1, FP2, F3, and F4, are selected.

[0049] The data enhancement includes sample segmentation and time-frequency transformation, as follows:

[0050] The EEG signal of the preprocessed data set is n seconds; a 1-second sliding window is used to cut the EEG signal in the time domain to significantly increase the number of samples;

[0051] For the input signal after sample segmentation, short-time Fourier transform is used to transform it into the frequency domain to extract features. The short-time Fourier transform formula is as follows:

[0052]

[0053] in is a window function, is the frequency, is a time index.

[0054] As an embodiment, in the public dataset DEAP dataset, the value of x is 3, the EEG signal of the preprocessed dataset is 60 seconds, and the final number of samples after segmenting the EEG signal using a 1-second time window through a sliding window method is 2400.

[0055] Step 2: Divide the dataset into the original training set, incremental training set and test set. The original training set is used for basic training, the incremental training set is used for incremental training, and the test set is used to evaluate the effect of the physiological-emotion recognition model.

[0056] The original training set is obtained by mixing the data of multiple individuals, and has a large enough data volume for pre-training; 5% of the data of the new individual is divided into an incremental training set for process incremental training, and the remaining 95% of the data of the new individual constitutes the test set to evaluate the model effect.

[0057] Step 3: Train the physiological-emotion recognition model (hereinafter referred to as TSN-LWF). The TSN-LWF model mainly consists of three modules, namely two parallel backbone networks (TSN1 and TSN2), two parallel output branches (N1 and N2) and an LWF-based classifier module. The training is divided into two stages. In the basic stage, the original training set is fed into the TSN-LWF model, the parameters of TSN1 and N1 are trained and the pre-trained model after convergence is saved. In the subsequent incremental stage, the output module N2 is randomly initialized, the parameters of the TSN1 module are migrated to TSN2, the TSN1 and N1 modules are fixed, and the incremental training set is input into the model. Only the TSN2 and N2 modules are trained. The LWF-based classifier calculates the loss and optimizes the model hyperparameters, and uses a public dataset for model training.

[0058] Among them, two parallel backbone networks (TSN1 and TSN2) are respectively connected to two parallel output modules (N1 and N2), the structures of TSN1 and TSN2 are the same, and the structures of N1 and N2 are the same; TSN1 and TSN2 are composed of a temporal convolution block, a spatial convolution block and a classifier module, N1 and N2 are both fully connected layers, and the output feature is m; the temporal convolution block is used to fully extract different frequency domain information, the spatial convolution block obtains a global feature representation by performing a convolution operation on the dimension of the EEG channel, and the classifier module is used to better obtain the emotion classification results.

[0059] like Figure 3 As shown in the figure, the temporal convolution block consists of three parallel multi-scale temporal convolutional layers. Each layer first performs a convolution operation, and the sizes of the convolution kernels are (1, 0.5*f), (1, 0.25*f), and (1, 0.15*f), respectively, where f refers to the number of sampling points. After the convolution, the RELU activation function and the average pooling layer are passed, and then the time-frequency features of different scales are connected along the feature dimension;

[0060] The spatial convolution block consists of two parallel multi-scale spatial convolution layers. Each layer first performs a convolution operation with kernel sizes of (0.5*c, 1) and (1*c, 1), where c refers to the number of electrodes. After the convolution, the RELU activation function, average pooling layer and batch normalization are used, and then spatial features of different scales are connected along the feature dimension.

[0061] The classifier module includes 3 ordinary convolution layers, 1 depthwise separable convolution layer and 2 connection layers; the input vector first passes through 3 ordinary convolution layers with the same convolution kernel size, and then passes through the depthwise separable convolution layer and obtains the output vector through the fully connected layer; the convolution kernel size of the ordinary convolution layer is 1×3, the step size is 1, the padding is 1, and the number of output channels is 32, 64, and 64 respectively; the depthwise separable convolution layer is composed of depthwise convolution and pointwise convolution, the depthwise convolution uses a 1x3 convolution kernel, and the pointwise convolution uses an output channel number of 128; after each convolution operation and depthwise separable convolution operation of the classifier module, batch normalization, RELU activation function, and maximum pooling are used to avoid model overfitting; the connection layer includes 2 fully connected layers with sizes of 512 and 128 respectively.

[0062] The LWF-based classifier module is as follows:

[0063] The LWF (Less Forgetting Learning) method calls the trained TSN1 and N1 models the teacher models, and TSN2 and N2 the student models. In the basic stage, the teacher model is trained. In the subsequent incremental stage, in order to allow the student model to learn from the teacher model and obtain higher recognition accuracy when learning new data, the samples in the incremental training set must first pass through two parallel backbone networks and two parallel output modules. After prediction, a regular loss is defined. and one by conventional loss and distillation losses The combined loss function , and update the model using the corresponding loss function at each stage.

[0064] The combined loss function is as follows:

[0065] Combined loss function By conventional loss and distillation losses composition,

[0066]

[0067] in is a network model, is the set of input data samples, is the corresponding true label, is the student model output, is the teacher model output, is the relaxation coefficient, is the set of parameters from the teacher model, are the student model parameters;

[0068] Regular losses To measure the degree to which a sample is correctly classified, a common multinomial logistic loss is used:

[0069]

[0070] in, is the corresponding true label, is the student model output,

[0071] Distillation loss Use KL divergence (Kullback-Leibler divergence, KLD for short) to evaluate the approximation between the two sets of probability distributions. By optimizing the KL divergence, the student model output is close to the recorded output of the teacher model, which will preserve the knowledge and transfer it to the new model.

[0072]

[0073] in and Represent the output of the teacher model and the student model respectively, is the distillation temperature;

[0074] In the basic stage, choose As the loss function, the parameters that need to be updated are the parameters in TSN1 and N1 , in the incremental phase, use As the loss function, the parameters of the output module N2 are randomly initialized , migrate TSN1 module parameters To TSN2, fix the parameters of TSN1 and N1 , and only update the parameters in TSN2 and N2, that is, .

[0075] As an example, in a binary classification task, the dimension of the input vector is ,in is the batch number, the value is 32, is the number of EEG channels, the value is 4, is the number of sampling points, which is 128, so the dimension of the input vector is The dimension of the output vector after the temporal convolution block is , the dimension of the output vector after the spatial convolution block is , the dimension of the output vector after the classifier module and the output branch is .in, The value is 0.1 and the value of T is 2.

[0076] Step 4: Output the results. Based on the TSN-LWF model trained in step 3, input the test set and output the emotion recognition results of cross-individual EEG signals in the binary or multi-classification tasks of valence and arousal dimensions. In the binary or multi-classification tasks, select 5 or the corresponding value as the threshold to divide each dimension into high / low levels.

[0077] The preferred embodiments of the present invention disclosed above are only used to help explain the present invention. The preferred embodiments do not describe all the details in detail, nor do they limit the invention to the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the present invention, so that those skilled in the art can understand and use the present invention well.

Claims

1. A method for cross-individual EEG signal emotion recognition based on incremental learning strategy, characterized in that: The following steps are involved: Step 1: import the collected EEG data and perform preprocessing and data enhancement on the data, wherein the preprocessing includes artifact removal, filtering and EEG channel selection, and the data enhancement includes sample segmentation and time-frequency transformation; Step 2: Divide the dataset into the original training set, incremental training set and test set. The original training set is used for basic training, the incremental training set is used for process incremental training, and the test set is used to evaluate the effect of the physiological-emotion recognition model. Step 3: Train the physiological-emotion recognition model (hereinafter referred to as TSN-LWF). The TSN-LWF model mainly consists of three modules, namely two parallel backbone networks (TSN1 and TSN2), two parallel output branches (N1 and N2) and an LWF-based classifier module; the training is divided into two stages. In the basic stage, the original training set is fed into the TSN-LWF model, the parameters of TSN1 and N1 are trained and the pre-trained model after convergence is saved; in the subsequent incremental stage, the output module N2 is randomly initialized, the parameters of the TSN1 module are migrated to TSN2, the TSN1 and N1 modules are fixed, the incremental training set is input into the model, and only the TSN2 and N2 modules are trained; the LWF-based classifier is to calculate the loss and optimize the model hyperparameters, and the model is trained using a public dataset; Step 4: Output the results. Based on the TSN-LWF model trained in step 3, input the test set and output the sentiment classification results of the valence and arousal dimensions in binary classification or multi-classification tasks. In the binary classification task, select 5 as the threshold to divide each dimension into high / low levels; in the multi-classification task, select a combination of each dimension for classification.

2. The method for emotion recognition based on EEG signals based on incremental learning strategy according to claim 1, characterized in that: The preprocessing includes artifact removal, filtering and EEG channel selection, specifically including: first, deleting the n-second pre-trial baseline of the EEG signal and downsampling the data to 128 Hz; then, using the independent component analysis method to remove the electrooculogram (EOG) mixed in the EEG signal; finally, in order to remove low-frequency and high-frequency noise interference, filtering the EEG signal with a 4-45 Hz bandpass filter; Existing physiological research results show that the frontal area stores more emotional information and is most closely related to emotions. The FP1-FP2 electrode pair and the F3-F4 electrode pair in the frontal area are located forward, in the area of ​​the forehead without hair obstruction, which is more conducive to placing dry electrodes to collect EEG signals. Therefore, the four electrode positions with the highest correlation with emotion recognition in the frontal area, namely FP1, FP2, F3, and F4, are selected.

3. The method for emotion recognition based on EEG signals based on incremental learning strategy according to claim 1, characterized in that: The data enhancement includes sample segmentation and time-frequency transformation, as follows: The processed EEG data set is n seconds long. A 1-second sliding window is used to cut the EEG signal in the time domain to significantly increase the number of samples. For the input signal after sample segmentation, short-time Fourier transform is used to transform it into the frequency domain to extract features. The short-time Fourier transform formula is as follows: in w ( n ) is the window function, w is the frequency, and t is the time index.

4. The method for emotion recognition based on EEG signals based on incremental learning strategy according to claim 1, characterized in that: In step 2, dividing the data set specifically includes: The dataset is divided into the original training set, incremental training set and test set. The original training set is obtained by mixing the data of multiple individuals and has a large enough data volume for pre-training. 5% of the data of the new individual is divided into an incremental training set for process incremental training, and the remaining 95% of the data of the new individual constitutes the test set to evaluate the model effect.

5. The method for EEG signal emotion recognition based on incremental learning strategy according to claim 1, characterized in that: In step 3, two parallel backbone networks (TSN1 and TSN2) are respectively connected to two parallel output modules (N1 and N2). The structures of TSN1 and TSN2 are the same, and the structures of N1 and N2 are the same. TSN1 and TSN2 are composed of a temporal convolution block, a spatial convolution block and a classifier module. Both N1 and N2 are fully connected layers, and the output feature is m. The temporal convolution block is used to fully extract different frequency domain information, and the spatial convolution block obtains a global feature representation by performing a convolution operation on the dimension of the EEG channel. The classifier module is used to better obtain the emotion classification result.

6. The method for EEG signal emotion recognition based on incremental learning strategy according to claim 5, characterized in that: The temporal convolution block consists of three parallel multi-scale temporal convolutional layers. Each layer first performs a convolution operation. The sizes of the convolution kernels are (1, 0.5*f), (1, 0.25*f), and (1, 0.15*f), respectively, where f refers to the number of sampling points. After the convolution, the RELU activation function and the average pooling layer are used, and then the time-frequency features of different scales are connected along the feature dimension. The spatial convolution block consists of two parallel multi-scale spatial convolution layers. Each layer first performs a convolution operation with kernel sizes of (0.5*c, 1) and (1*c, 1), where c refers to the number of electrodes. After the convolution, the RELU activation function, average pooling layer and batch normalization are used, and then spatial features of different scales are connected along the feature dimension. The dimension of the input vector is defined as The dimension of the output vector is 1 is the number of convolution channels, c is the number of electrodes, f is the time sampling point, ch is the number of features after the convolution layer, and n is the number of features after the pooling operation; first, the time convolution block is used to learn the time-frequency representation, and then the spatial convolution block works in a cross-channel manner to obtain the spatial representation. The classifier module includes 3 common convolutional layers, 1 depth-separable convolutional layer and 2 connection layers; The input vector first passes through three ordinary convolution layers with the same convolution kernel size, and then passes through the depthwise separable convolution layer and the fully connected layer to obtain the output vector; the convolution kernel size of the ordinary convolution layer is 1×3, the step size is 1, the padding is 1, and the number of output channels is 32, 64, and 128 respectively; the depthwise separable convolution layer is composed of depthwise convolution and pointwise convolution. The depthwise convolution uses a 1x3 convolution kernel, and the pointwise convolution uses 64 output channels; after each convolution operation and depthwise separable convolution operation of the classifier module, batch normalization, RELU activation function, and maximum pooling are used to avoid overfitting of the model; the connection layer includes 2 fully connected layers with sizes of 512 and 128 respectively.

7. The method for EEG signal emotion recognition based on incremental learning strategy according to claim 1, characterized in that: In step 3, the LWF-based classifier module is as follows: The LWF method calls the trained TSN1 and N1 models the teacher models, and TSN2 and N2 the student models. In the basic stage, the teacher model is trained. In the subsequent incremental stage, in order to allow the student model to learn from the teacher model and obtain higher recognition accuracy when learning new data, the samples in the incremental training set must first pass through two parallel backbone networks and two parallel output modules. After prediction, a conventional loss L is defined. new and a loss L new and distillation loss L old The combined loss function L LWF , and update the model using the corresponding loss function at each stage.

8. The method for EEG signal emotion recognition based on incremental learning strategy according to claim 8, characterized in that: The combined loss function is as follows: Combined loss function L LWF By conventional loss L new and distillation loss L old composition, L LWF =L new (and n ,y)+λL old (and n ,and o ), in is the network model, x is the set of input data samples, y is the corresponding true label, y n is the student model output, y0 is the teacher model output, λ is the relaxation coefficient of KDL, θ0 is the parameter set from the teacher model, and θ is the student model parameter; Conventional loss L new To measure the degree to which a sample is correctly classified, a common multinomial logistic loss is used: L new (y n ,y)=-y·logy n Among them, y is the corresponding true label, y n is the student model output, Distillation loss L old Use KL (Kullback-Leibler) divergence to evaluate the approximation between the two sets of probability distributions. By optimizing the KL divergence, the student model output is close to the recorded output of the teacher model, which will preserve the knowledge and transfer it to the new model. where y0 and y n denote the outputs of the teacher model and the student model respectively, T is the distillation temperature; In the basic stage, choose L new As the loss function, the parameters that need to be updated are the parameters θ in TSN1 and out1 s ,θ o , in the incremental phase, use L LWF As the loss function, the parameters θ of the output module out2 are randomly initialized n , migrate TSN1 module parameters θ s To TSN2, fix the parameters θ of TSN1 and out1 s ,θ o , and only update the parameters in TSN2 and out2, i.e., θ s ,θ n →θ' s ,θ' n .

Images