Emotion recognition model training method

Abstract

The invention relates to a training method of an emotion recognition model, and belongs to the field of signal processing. Comprising the following steps: constructing a discrete emotion set and a video material database, and collecting electroencephalogram signals when a subject watches a video in the video material database; preprocessing the electroencephalogram signal, obtaining a four-dimensional brain map based on the preprocessed electroencephalogram signal, labeling the four-dimensional brain map based on the discrete emotion set, and constructing a data set based on the labeled four-dimensional brain map; and constructing an emotion recognition model, and determining parameters of the emotion recognition model based on a genetic algorithm and the data set to obtain a trained emotion recognition model. According to the method, frequency domain, space domain and time domain information of electroencephalogram signals is integrated through the four-dimensional brain map, parameters of the emotion recognition model are optimized through the genetic algorithm, the emotion recognition accuracy is improved, and the problems that an existing emotion recognition model is low in recognition fine grit, and particularly the recognition accuracy of materials with small emotion spans is poor are solved.

Description

Technical Field

[0001] The present invention relates to the technical field of signal processing, and in particular to a training method for an emotion recognition model. Background Art

[0002] Emotions are human responses to daily interactions. Accurately identifying the audience's emotions helps adjust the content and communication strategies of media production, and plays a vital role in improving the effectiveness of media communication. In recent years, EEG signals have been widely used in emotion recognition due to their ease of use and non-invasiveness.

[0003] Currently existing emotion recognition models usually recognize EEG signals when subjects watch videos with a large emotional span, such as videos of harvest and earthquake, so that emotions can be more easily recognized. However, in many scenarios, especially in the process of media communication, the emotional span of videos is small, and how to achieve fine-grained recognition of corresponding emotions is still the main challenge that needs to be faced. Secondly, existing emotion recognition models usually only recognize based on one or two of the frequency domain, spatial domain and time domain features of EEG signals, which may result in the loss of key emotional information and reduce the accuracy of emotion recognition. Finally, existing emotion recognition methods based on EEG signals usually only classify the predicted emotions into positive, negative and neutral, and the recognition granularity is low.

[0004] It can be seen that building an emotion recognition model based on EEG signals to achieve fine-grained and accurate recognition of corresponding emotions for materials with a smaller emotional span is a problem that needs to be faced. Summary of the invention

[0005] In view of the above analysis, the present invention aims to provide a training method for an emotion recognition model to solve the problem that the emotion recognition model trained by the current training method cannot accurately recognize emotions for video materials with a small emotional span and has a low recognition granularity.

[0006] The present invention provides a method for training an emotion recognition model, the method comprising the following steps:

[0007] Constructing a discrete emotion set and a video material database, and collecting EEG signals of subjects when they watch videos in the video material database;

[0008] Preprocessing the EEG signal, obtaining a four-dimensional brain map based on the preprocessed EEG signal, annotating the four-dimensional brain map based on the discrete emotion set, and constructing a data set based on the annotated four-dimensional brain map;

[0009] An emotion recognition model is constructed, and parameters of the emotion recognition model are determined based on a genetic algorithm and the data set to obtain a trained emotion recognition model.

[0010] Furthermore, determining the parameters of the emotion recognition model based on the genetic algorithm and the data set to obtain a trained emotion recognition model includes:

[0011] Step S301, dividing the data set into a training set and a test set in a ratio of 8:2;

[0012] Step S302, initializing configuration parameters, and generating genetic individuals based on the configuration parameters to construct a population;

[0013] Step S303, calculating the fitness value of each genetic individual in the population, sorting all genetic individuals from small to large according to their fitness values, and selecting the genetic individuals in the first half of the sorting to form an excellent population;

[0014] Step S304, determine whether the termination condition is met. If so, obtain the corresponding trained emotion recognition model based on the genetic individual with the smallest fitness value in the population; otherwise, crossover and mutation are performed according to the crossover probability and mutation probability to obtain new genetic individuals, and an updated population is obtained based on the genetic individuals in the excellent population and the new genetic individuals. The updated population is used as the population for the next iteration, and steps S303 to S304 are repeated.

[0015] Furthermore, the calculation of the fitness value of each genetic individual in the population includes:

[0016] Based on the characteristic value of each genetic individual, an emotion recognition model corresponding to each genetic individual is constructed, and the learning rate, regularization parameter alpha, and maximum number of iterations are set;

[0017] Using the training set and the learning rate, the regularization parameter alpha, and the maximum number of iterations to train the emotion recognition model corresponding to each genetic individual, outputting the predicted discrete emotions, and obtaining the trained emotion recognition model corresponding to each genetic individual when the loss function reaches a threshold or the iteration reaches a maximum number of iterations;

[0018] The test set is predicted using the trained emotion recognition model corresponding to each genetic individual, the accuracy is calculated based on the prediction result, and the inverse of the accuracy is used as the fitness value of each genetic individual.

[0019] Furthermore, each genetic individual has 4 eigenvalues, wherein the first eigenvalue is the number of neurons in the first layer of the hidden layer, the second eigenvalue is the number of neurons in the second layer of the hidden layer, the third eigenvalue is the type of activation function, and the fourth eigenvalue is the type of optimizer.

[0020] Furthermore, if the termination condition is not reached, a pair of parent genetic individuals is established based on the population, and the second eigenvalue and the fourth eigenvalue of the parent genetic individual pair are exchanged according to the crossover probability to generate a new genetic individual; the first eigenvalue or the second eigenvalue is randomly selected to generate a random number between [1,4], and the selected eigenvalue plus the random number is mutated as a new eigenvalue according to the mutation probability to generate a new genetic individual; the fitness value of the new genetic individual is calculated; the genetic individuals in the excellent population are combined with the new genetic individuals, and are selected according to the number of genetic individuals in the population and the fitness values ​​of the genetic individuals in the combination to obtain an updated population.

[0021] Furthermore, the establishing of parental genetic individual pairs based on population includes:

[0022] The first half of the genetic individuals in the sequence are selected to constitute the first parent generation, and the second half of the genetic individuals are selected to constitute the second parent generation;

[0023] The genetic individuals in the first parent generation and the second parent generation form several parent generation genetic individual pairs in pairs according to the sequence numbers.

[0024] Furthermore, all genetic individuals in the combination are sorted from small to large according to their fitness values, and the first N genetic individuals whose number is the same as that of the genetic individuals in the population are selected from the sorting to obtain an updated population.

[0025] Furthermore, the emotions that appear when the subjects watch the videos in the video material database are divided to construct a discrete emotion set; each discrete emotion in the discrete emotion set has high or low pleasure, high or low arousal attributes.

[0026] Furthermore, after watching the currently selected video, the current subject makes an emotional evaluation on the video, that is, selects a corresponding discrete emotion from the discrete emotion set.

[0027] Furthermore, the four-dimensional brain maps corresponding to all the subjects are annotated based on the emotion evaluation; wherein, the annotated four-dimensional brain map corresponding to a subject watching a video is a sample, and the annotated four-dimensional brain maps corresponding to all the subjects constitute a data set.

[0028] Compared with the prior art, the present invention can achieve at least one of the following beneficial effects:

[0029] 1. The present invention converts the EEG signal into a four-dimensional data block containing its frequency domain, spatial domain and time domain information through a four-dimensional brain map, retains the key emotional information in the EEG signal, and improves the accuracy of emotion recognition.

[0030] 2. The present invention determines the parameters of the emotion recognition model based on a genetic algorithm, thereby obtaining a trained emotion recognition model, optimizing the parameters of the model, and improving the accuracy of emotion recognition.

[0031] 3. The present invention collects videos with a small emotional span to construct a video material database, collects EEG signals based on the video material database and constructs a corresponding data set, and trains the emotion recognition model based on the data set, thereby solving the problem of achieving fine-grained and accurate recognition of corresponding emotions for materials with a small emotional span, and providing a basis for improving the effectiveness of media communication.

[0032] 4. The present invention improves the accuracy of emotion recognition by extracting differential entropy and power spectral density of EEG signals as features.

[0033] 5. The present invention divides the emotions that appear when the subjects watch the videos in the video material database, constructs a discrete emotion set, and covers complex emotions intertwined with multiple emotions, so that the subjects can express their emotional feelings more accurately, providing a basis for achieving fine-grained and accurate recognition of emotions.

[0034] In the present invention, the above-mentioned technical solutions can also be combined with each other to achieve more preferred combination solutions. Other features and advantages of the present invention will be described in the subsequent description, and some advantages can become obvious from the description, or can be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained through the contents particularly pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The drawings are only for the purpose of illustrating particular embodiments and are not to be considered limiting of the present invention. Like reference symbols denote like components throughout the drawings.

[0036] Figure 1 is a flow chart of a method for training an emotion recognition model according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of nine videos selected for an embodiment of the present invention;

[0038] Figure 3 A schematic diagram of a mind map creation process according to an embodiment of the present invention;

[0039] Figure 4 A schematic diagram of performance comparison of different classifier models according to an embodiment of the present invention;

[0040] Figure 5 Schematic diagram of an ablation experiment on different input features according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings, wherein the accompanying drawings constitute a part of this application and are used together with the embodiments of the present invention to illustrate the principles of the present invention, but are not used to limit the scope of the present invention.

[0042] A specific embodiment of the present invention discloses a method for training an emotion recognition model.

[0043] like Figure 1 As shown, the method comprises the following steps:

[0044] Step S1, constructing a discrete emotion set and a video material database, and collecting EEG signals of subjects when they watch videos in the video material database;

[0045] Step S2, preprocessing the EEG signal, obtaining a four-dimensional brain map based on the preprocessed EEG signal, annotating the four-dimensional brain map based on the discrete emotion set, and constructing a data set based on the annotated four-dimensional brain map;

[0046] Step S3: construct an emotion recognition model, and determine the parameters of the emotion recognition model based on the genetic algorithm and the data set to obtain a trained emotion recognition model.

[0047] Specifically, in step S1, videos with a small emotional span are collected to construct a video material database, videos are randomly selected from the video material database, and the subjects watch the selected videos while collecting their EEG signals.

[0048] For example, videos about Chinese stories are collected to construct a video material database, and the emotional span of the relevant videos is relatively small. It should be noted that videos can be selected to construct a video material database according to actual needs or actual application scenarios.

[0049] Specifically, when watching the selected video, the subject first rests for a period of time, and then watches the selected video. The subject's EEG signal is collected by an EEG signal collector, which includes an EEG (Electroencephalogram) electrode cap and an EEG signal recorder; the EEG electrode cap is distributed according to the international 10-20 standard, and the ground electrode is FPz; the sampling frequency of the EEG signal recorder is 1000Hz.

[0050] It should be noted that there are several subjects and selected videos, and each subject needs to watch all the selected videos.

[0051] It is understandable that currently available emotion recognition models usually recognize EEG signals when subjects watch videos with a large emotional span, such as videos of harvest and earthquake. These videos have obvious emotional orientation, making emotions easier to recognize. However, the present invention aims to solve the problem of constructing an emotion recognition model based on EEG signals to achieve fine-grained and accurate recognition of corresponding emotions for materials with a small emotional span. Therefore, the present invention collects videos with a small emotional span to construct a video material database, collects EEG signals based on the video material database and constructs a corresponding data set, and trains the emotion recognition model based on the data set, thereby solving the aforementioned problems and providing a basis for improving the effectiveness of media communication.

[0052] Furthermore, the emotions that appear when the subjects watch the videos in the video material database are divided to construct a discrete emotion set; the discrete emotion set includes eight discrete emotions.

[0053] Specifically, each discrete emotion in the discrete emotion set represents a specific emotion type, and each discrete emotion has the attributes of pleasure and arousal. The pleasure represents the positivity of an emotion, with high pleasure representing positive emotions and low pleasure representing negative emotions; the arousal represents the intensity of an emotion, with high arousal representing strong emotions, such as excitement or tension, and low arousal representing calm emotions, such as relaxation or fatigue. Each discrete emotion has the attributes of high or low pleasure and high or low arousal. After watching the videos in the video material database, the subjects scored the emotions that appeared, divided the emotions that appeared based on the scoring results, and constructed a discrete emotion set.

[0054] Exemplarily, the emotions that appear when the subjects watch videos in a video material database constructed from Chinese stories are divided to construct a discrete emotion set, which includes eight discrete emotions: admired, proud, happy, surprised, introspective, relaxed, bored, and touched.

[0055] It can be understood that in the above example, since the videos related to Chinese stories are all neutral to positive emotions, the emotions in the discrete emotion set are all neutral to positive.

[0056] It should be noted that after the subjects watched the videos in the video material database, they scored the basic discrete emotions according to their own emotions. These basic discrete emotions constitute the basis of all emotions. If the emotional type classification is unreasonable, it will lead to overlaps between different emotional types, or some emotions cannot be classified into any emotional type. Since the present invention aims to achieve fine-grained and accurate recognition of emotions, the present invention divides the emotions that appear in the videos watched by the subjects in the video material database, constructs a discrete emotion set, and subdivides the emotions into eight discrete emotions, which include complex emotions intertwined with multiple emotions, so that the subjects can express their emotional feelings more accurately, providing a basis for achieving fine-grained and accurate recognition of emotions.

[0057] Furthermore, after watching the currently selected video, the current subject makes an emotional evaluation on the video, that is, selects a corresponding discrete emotion from the discrete emotion set.

[0058] Exemplarily, 9 videos are randomly selected from the video material database such as Figure 2 As shown in the figure, each video lasts 2-2.5 minutes, and each subject completes watching the 9 videos. When a subject watches each video, the subject first rests for one minute to stabilize the EEG signal, and then watches the video; after watching the video, the subject makes an emotional evaluation, and then watches the next video.

[0059] Specifically, in step S2, the preprocessing of the EEG signal includes:

[0060] The EEG signals corresponding to each subject watching the video are extracted through bandpass filtering and downsampled;

[0061] Extract bad electrodes and interpolate the downsampled EEG signals;

[0062] The interpolated EEG signals were de-artifacted and re-referenced by whole-brain average;

[0063] The EEG signal after re-reference is intercepted.

[0064] Specifically, the EEG signal corresponding to each subject watching the video was extracted through a 0.5-49 Hz bandpass filter and downsampled to 128 Hz.

[0065] It should be noted that the EEG signals of all subjects were preprocessed. Since the subjects first rested for a period of time and then watched the video when watching the video, the collected EEG signals included the EEG signals corresponding to resting and watching the video. Through the above-mentioned bandpass filtering, the EEG signals corresponding to the subjects watching the video can be effectively extracted.

[0066] Furthermore, bad electrodes are extracted from the downsampled EEG signal, and the extracted bad electrodes are saved in a data list. The bad electrodes include the names of electrodes marked as bad electrodes.

[0067] It should be noted that bad electrodes refer to poor brain electrode data caused by improper operation during the acquisition process, whose amplitude exceeds the upper or lower threshold, and the bad electrodes are screened out through the threshold.

[0068] Specifically, for each bad electrode in the data list:

[0069] According to the name of its electrode, get the name of its location and the electrodes around it;

[0070] Determine whether the electrodes around it are bad electrodes according to the names of the electrodes around it;

[0071] The bad electrode is interpolated based on the amplitudes of the surrounding non-bad electrodes.

[0072] Specifically, according to the international 10-20 standard distribution, the position of the bad electrode can be obtained through the electrode name of the bad electrode, and the names of the electrodes around it can be obtained through the position of the bad electrode. The names of the electrodes around it are searched in the data list in sequence. If the names are found, the corresponding electrode is judged to be a bad electrode, otherwise it is a good electrode. The bad electrode is interpolated based on the average value of the amplitudes of the non-bad electrodes around it.

[0073] Furthermore, the interpolated EEG signal was filtered, and the artifact component was obtained and removed through ICA (Independent Component Analysis), and the whole-brain average value was re-referenced.

[0074] Specifically, the interpolated EEG signal is band-pass filtered from 8 to 35 Hz to remove low-frequency drift and high-frequency noise. The number of ICA components is set, and the artifact component is obtained and removed through ICA. The average value of the EEG signal of all electrodes in the whole brain after artifact removal is used as the reference value.

[0075] It should be noted that artifacts refer to EEG signal noise or artifacts caused by environmental factors, blinking, muscle tension, etc. These artifacts will seriously interfere with the quality of EEG signals and affect the accuracy of emotion recognition, so they need to be removed. ICA can identify artifact components by decomposing EEG signals. The present invention calls the ICA algorithm through the MNE-Python library to remove artifacts. Re-referencing the whole brain average can reduce artifacts caused by different reference electrode positions.

[0076] Furthermore, the EEG signal after re-reference is intercepted, for example, the data of the last 100 seconds is retained.

[0077] Furthermore, the preprocessed EEG signal is segmented in the time domain, divided in the frequency domain, extracted in the feature domain, and mapped in the space domain to obtain a two-dimensional atlas, and the two-dimensional atlas is superimposed in the frequency domain and in the time domain to obtain a four-dimensional brain map.

[0078] Specifically, the preprocessed EEG signal is divided into a series of non-overlapping time segments of length Ts in the time domain; each time segment is divided into four frequency bands of alpha, beta, gamma and theta in the frequency domain; for each frequency band, the differential entropy (DE) and power spectral density (PSD) of the EEG signal are extracted; for each frequency band, the corresponding differential entropy and power spectral density are respectively mapped to a two-dimensional grid corresponding to the position of the EEG signal electrode for spatial domain mapping, and a two-dimensional spectrum corresponding to each frequency band is obtained.

[0079] Specifically, the above-mentioned frequency bands have different frequency ranges and represent different neural activity information. The frequency range of the alpha band is 8-13Hz, representing the state of a person when awake, quiet and with eyes closed; the frequency range of the beta band is 13-30Hz, representing the state of a person when the mind is active, alert and nervous; the frequency range of the theta band is 4-8Hz, representing the state of a person when sleeping, relaxing and meditating; the frequency range of the gamma band is 30-100Hz, representing the state of a person when the brain is highly active, cognitive and perceptual. For each time segment, according to the frequency range of the frequency band, the corresponding Butterworth bandpass filter is applied, and the frequency domain is divided into four key frequency bands of alpha, beta, gamma and theta, that is, the number of frequency bands is equal to 4. For each frequency band, the differential entropy of the EEG signal is obtained, and the energy of the EEG signal is obtained as its power spectral density using a sliding time window of 0.5 seconds. Figure 3 As shown, the two-dimensional grid is an 8×9 spatial layout, wherein each grid point represents the position of an EEG signal electrode that meets the international 10-20 standard; there are two two-dimensional spectra corresponding to each frequency band, including a differential entropy two-dimensional spectrum and a power spectrum density two-dimensional spectrum; the differential entropy two-dimensional spectrum and the power spectrum density two-dimensional spectrum corresponding to each frequency band are obtained respectively by the differential entropy and power spectrum density corresponding to each frequency band, therefore, there are two two-dimensional spectra corresponding to each frequency band; each two-dimensional spectrum represents an 8×9 data block.

[0080] It should be noted that each electrode is an acquisition channel, and the corresponding differential entropy and power spectral density are extracted from the EEG signal collected by each electrode, and then the extracted differential entropy and power spectral density are respectively mapped to the grid points corresponding to the electrode.

[0081] Furthermore, the differential entropy is obtained by the following formula:

[0082]

[0083] Among them, h(x i ) represents x i The differential entropy of x i represents the EEG signal corresponding to the ith frequency band, σ 2 Represents x i The variance of , e represents the base of the natural logarithm.

[0084] Furthermore, the energy is obtained by the following formula:

[0085]

[0086] Where E represents x i The energy of ij represents the EEG signal corresponding to the jth sliding time window in the ith frequency band, and N represents the number of sliding time windows.

[0087] It should be noted that the power spectrum density of the EEG signal can usually be approximately represented by energy. The present invention improves the accuracy of emotion recognition by extracting the differential entropy and power spectrum density of the EEG signal as features.

[0088] Furthermore, the two-dimensional maps corresponding to each frequency band are superimposed, and then each time segment is superimposed to obtain a four-dimensional brain map.

[0089] Specifically, Figure 3 As shown, the eight two-dimensional maps corresponding to the four frequency bands are superimposed to obtain an 8×9×8 data block; then the time segments are superimposed to obtain a four-dimensional brain map, which represents a data blocks, where T represents the total duration of the preprocessed EEG signal, for example, 100 seconds, and Ts is the length of the time segment. Represents the number of time slices.

[0090] It can be understood that the present invention converts the EEG signal into a four-dimensional data block containing its frequency domain, spatial domain and time domain information through a four-dimensional brain map, retains the key emotional information in the EEG signal, and improves the accuracy of emotion recognition.

[0091] It should be noted that the four-dimensional brain maps corresponding to all subjects were obtained through the above steps.

[0092] Specifically, the four-dimensional brain maps corresponding to all the subjects are annotated based on the emotion evaluation; wherein the annotated four-dimensional brain map corresponding to a subject watching a video is a sample, and the annotated four-dimensional brain maps corresponding to all the subjects constitute a data set.

[0093] It should be noted that the emotional evaluation of a video by the subject includes selecting one or more discrete emotions, because when watching a video, multiple emotions may appear at the same time. For example, when watching a video of a rocket launch, emotions such as pride, happiness, and emotion may appear at the same time. Therefore, the labels of the four-dimensional brain map corresponding to the subject watching a video can be one or more.

[0094] Specifically, in step S3, a multilayer perceptron is constructed as an emotion recognition model, wherein the number of hidden layers is 2; the parameters of the emotion recognition model include the number of neurons in each layer of the hidden layer, the type of activation function, and the type of optimizer.

[0095] It should be noted that the multilayer perceptron, as a classifier, includes an input layer, a hidden layer, and an output layer; each node in the hidden layer and the output layer undergoes a linear transformation, and an activation function is applied to map the weighted sum of the neurons in the previous layer to the output for classification.

[0096] Furthermore, determining the parameters of the emotion recognition model based on the genetic algorithm and the data set to obtain a trained emotion recognition model includes:

[0097] Step S301, dividing the data set into a training set and a test set in a ratio of 8:2;

[0098] Step S302, initializing configuration parameters, and generating genetic individuals based on the configuration parameters to construct a population;

[0099] Step S303, calculating the fitness value of each genetic individual in the population, sorting all genetic individuals from small to large according to their fitness values, and selecting the genetic individuals in the first half of the sorting to form an excellent population;

[0100] Step S304, determine whether the termination condition is met. If so, obtain the corresponding trained emotion recognition model based on the genetic individual with the smallest fitness value in the population; otherwise, crossover and mutation are performed according to the crossover probability and mutation probability to obtain new genetic individuals, and an updated population is obtained based on the genetic individuals in the excellent population and the new genetic individuals. The updated population is used as the population for the next iteration, and steps S303 to S304 are repeated.

[0101] Specifically, in step S302, the configuration parameters include the number of genetic individuals in the population, the number of neurons in the first layer of the hidden layer, the number of neurons in the second layer of the hidden layer, the type of activation function, the type of optimizer, the crossover probability and the mutation probability. Genetic individuals are generated based on the number of neurons in the first layer of the hidden layer, the number of neurons in the second layer of the hidden layer, the type of activation function, the type of optimizer, that is, the parameters of the emotion recognition model. The number of neurons in the first layer of the hidden layer is an integer between [2,100], and the number of neurons in the second layer of the hidden layer is an integer between [2,50]; the type of activation function is one of identity (identity function), logistic (logistic function), tanh (hyperbolic tangent function), relu (linear rectified unit function); the type of optimizer is one of lbfgs (L-BFGS algorithm optimizer), sgd (stochastic gradient descent optimizer), and adam (Adam optimizer). The initial population is constructed by randomly selecting corresponding data within the above value range to generate genetic individuals; each genetic individual has 4 eigenvalues, among which the first eigenvalue is the number of neurons in the first layer of the hidden layer, the second eigenvalue is the number of neurons in the second layer of the hidden layer, the third eigenvalue is the type of activation function, and the fourth eigenvalue is the type of optimizer.

[0102] It should be noted that the number of genetic individuals in the population is an even number, which is convenient for subsequent crossover and mutation operations. The set crossover probability and mutation probability should ensure that the number of new genetic individuals obtained through crossover and mutation is greater than or equal to half of the number of genetic individuals in the population.

[0103] For example, the number of genetic individuals in the population is set to 40, the crossover probability is 0.95, and the mutation probability is 0.05. A genetic individual is generated as: X = [46, 16, relu, sgd].

[0104] Specifically, in step S303, the calculation of the fitness value of each genetic individual in the population includes:

[0105] Based on the parameters of each genetic individual, an emotion recognition model corresponding to each genetic individual is constructed, and the learning rate, regularization parameter alpha, and maximum number of iterations are set;

[0106] Using the training set and the learning rate, the regularization parameter alpha, and the maximum number of iterations to train the emotion recognition model corresponding to each genetic individual, outputting the predicted discrete emotions, and obtaining the trained emotion recognition model corresponding to each genetic individual when the loss function reaches a threshold or the iteration reaches a maximum number of iterations;

[0107] The test set is predicted using the trained emotion recognition model corresponding to each genetic individual, the accuracy is calculated based on the prediction result, and the inverse of the accuracy is used as the fitness value of each genetic individual.

[0108] It should be noted that the accuracy is the ratio of the number of correctly predicted samples to the total number of samples.

[0109] For example, the learning rate is set to 0.5 and alpha = 1×e -5 , the maximum number of iterations is 1000.

[0110] Preferably, in order to make the emotion recognition model converge quickly, the maximum number of iterations when there is no improvement is set to 80. That is, if the accuracy rate does not improve after 80 iterations, the training is stopped.

[0111] Specifically, in step S304, the termination condition includes that the fitness value of a genetic individual in the current population is less than a threshold or the number of iterations reaches a maximum value.

[0112] Furthermore, if the termination condition is not reached, a pair of parent genetic individuals is established based on the population, and the second eigenvalue and the fourth eigenvalue of the parent genetic individual pair are exchanged according to the crossover probability to generate a new genetic individual; the first eigenvalue or the second eigenvalue is randomly selected to generate a random number between [1,4], and the selected eigenvalue plus the random number is mutated as a new eigenvalue according to the mutation probability to generate a new genetic individual; the fitness value of the new genetic individual is calculated; the genetic individuals in the excellent population are combined with the new genetic individuals, and are selected according to the number of genetic individuals in the population and the fitness values ​​of the genetic individuals in the combination to obtain an updated population.

[0113] Specifically, the establishing of parental genetic individual pairs based on population includes:

[0114] The first half of the genetic individuals in the sequence are selected to constitute the first parent generation, and the second half of the genetic individuals are selected to constitute the second parent generation;

[0115] The genetic individuals in the first parent generation and the second parent generation form several parent generation genetic individual pairs in pairs according to the sequence numbers.

[0116] It should be noted that the sorting is to sort all genetic individuals in the population from small to large according to their fitness values.

[0117] For example, assume that the number of genetic individuals in the population is 6, and the 6 genetic individuals are: [62,22,relu,sgd], [30,39,logistic,adam], [20,40,tanh,lbfgs], [8,14,identity,adam], [25,37,relu,lbfgs], [35,28,tanh,sgd], and the fitness values ​​of the 6 genetic individuals are: 0.725, 0.65, 0.50, 0.375, 0.63, and 0.60. The genetic individuals in the first parent generation are arranged in order: [62,22,relu,sgd], [30,39,logistic,adam], [25,37,relu,lbfgs], among which the sequence number of [62,22,relu,sgd] is 1; the genetic individuals in the second parent generation are arranged in order: [35,28,tanh,sgd], [20,40,tanh,lbfgs], [8,14,identity,adam]. The genetic individuals in the first and second parent generations are arranged in sequence to form three pairs of parent genetic individual pairs: [[62,22,relu,sgd]-[35,28,tanh,sgd]], [[30,39,logistic,adam]-[20,40,tanh,lbfgs]], and [[25,37,relu,lbfgs]-[8,14,identity,adam]].

[0118] Specifically, for each pair of parent genetic individuals, the second eigenvalue of the first parent is exchanged with the second eigenvalue of the second parent, and the fourth eigenvalue of the first parent is exchanged with the fourth eigenvalue of the second parent according to the crossover probability to generate a new genetic individual.

[0119] Exemplarily, the crossover operation is performed on the first pair of parent genetic individuals to generate new genetic individuals: [62, 28, relu, sgd], [35, 22, tanh, sgd].

[0120] Specifically, for each pair of parent genetic individuals, the selected characteristic value plus the random number is used as a new characteristic value for mutation according to the mutation probability to generate a new genetic individual.

[0121] For example, assuming that the first eigenvalue is randomly selected and a random number 3 is generated between [1,4], the above-mentioned first pair of parent genetic individuals is mutated to generate new genetic individuals: [65,22,relu,sgd], [38,28,tanh,sgd].

[0122] It should be noted that if the mutated eigenvalue exceeds the maximum value of the eigenvalue, the mutated eigenvalue is set to the maximum value of the eigenvalue.

[0123] Specifically, all genetic individuals in the combination are sorted from small to large according to their fitness values, and the first N genetic individuals whose number is the same as that of the genetic individuals in the population are selected from the sorting to obtain an updated population.

[0124] It should be noted that, during implementation, the above method is used to collect EEG signals of the subjects when they watch the videos in the video material database, the EEG signals are preprocessed, and a four-dimensional brain map is obtained based on the preprocessed EEG signals. The four-dimensional brain map is input into the trained emotion recognition model to obtain predicted discrete emotions, wherein each predicted discrete emotion has pleasantness and arousal attributes.

[0125] Performance evaluation:

[0126] like Figure 4 As shown in Figure 2, the performance of the emotion recognition model and other common classifier models are evaluated from three aspects: accuracy, precision, and recall. Figure 4 It can be seen that the recognition results of other common classifier models for the 8 discrete emotions in the present invention are all less than 50%, while the recognition accuracy of the emotion recognition model provided by the present invention is 82.24%, which is much better than other models. This is because the emotional span of the video selected by the present invention is relatively small, which greatly increases the difficulty of emotion recognition, and it is difficult for existing models to accurately classify it.

[0127] like Figure 5 As shown in Figure 2, ablation experiments are conducted on different input features to evaluate the accuracy of different features in emotion recognition tasks. Figure 5 It can be seen that compared with using only a single time domain feature or frequency domain feature, the four-dimensional brain map combining frequency domain, time domain and spatial domain features has a higher accuracy of emotion recognition.

[0128] Compared with the prior art, the training method of the emotion recognition model provided by the present invention has the following beneficial effects:

[0129] 1. The present invention converts the EEG signal into a four-dimensional data block containing its frequency domain, spatial domain and time domain information through a four-dimensional brain map, retains the key emotional information in the EEG signal, and improves the accuracy of emotion recognition.

[0130] 2. The present invention determines the parameters of the emotion recognition model based on a genetic algorithm, thereby obtaining a trained emotion recognition model, optimizing the parameters of the model, and improving the accuracy of emotion recognition.

[0131] 3. The present invention collects videos with a small emotional span to construct a video material database, collects EEG signals based on the video material database and constructs a corresponding data set, and trains the emotion recognition model based on the data set, thereby solving the problem of achieving fine-grained and accurate recognition of corresponding emotions for materials with a small emotional span, and providing a basis for improving the effectiveness of media communication.

[0132] 4. The present invention improves the accuracy of emotion recognition by extracting differential entropy and power spectral density of EEG signals as features.

[0133] 5. The present invention divides the emotions that appear when the subjects watch the videos in the video material database, constructs a discrete emotion set, and covers complex emotions intertwined with multiple emotions, so that the subjects can express their emotional feelings more accurately, providing a basis for achieving fine-grained and accurate recognition of emotions.

[0134] Those skilled in the art will appreciate that all or part of the processes of the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, wherein the computer-readable storage medium is a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0135] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by any technician familiar with the technical field within the technical scope disclosed by the present invention should be covered within the protection scope of the present invention.

Claims

1. A training method for an emotion recognition model, characterized in that: The method comprises the following steps: Constructing a discrete emotion set and a video material database, and collecting EEG signals of subjects when they watch videos in the video material database; Preprocessing the EEG signal, obtaining a four-dimensional brain map based on the preprocessed EEG signal, annotating the four-dimensional brain map based on the discrete emotion set, and constructing a data set based on the annotated four-dimensional brain map; An emotion recognition model is constructed, and parameters of the emotion recognition model are determined based on a genetic algorithm and the data set to obtain a trained emotion recognition model.

2. The method for training an emotion recognition model according to claim 1, characterized in that: The step of determining the parameters of the emotion recognition model based on the genetic algorithm and the data set to obtain a trained emotion recognition model comprises: Step S301, dividing the data set into a training set and a test set in a ratio of 8:2; Step S302, initializing configuration parameters, and generating genetic individuals based on the configuration parameters to construct a population; Step S303, calculating the fitness value of each genetic individual in the population, sorting all genetic individuals from small to large according to their fitness values, and selecting the genetic individuals in the first half of the sorting to form an excellent population; Step S304, determine whether the termination condition is met. If so, obtain the corresponding trained emotion recognition model based on the genetic individual with the smallest fitness value in the population; otherwise, crossover and mutation are performed according to the crossover probability and mutation probability to obtain new genetic individuals, and an updated population is obtained based on the genetic individuals in the excellent population and the new genetic individuals. The updated population is used as the population for the next iteration, and steps S303 to S304 are repeated.

3. The method for training an emotion recognition model according to claim 2, characterized in that: The calculation of the fitness value of each genetic individual in the population includes: Based on the characteristic value of each genetic individual, an emotion recognition model corresponding to each genetic individual is constructed, and the learning rate, regularization parameter alpha, and maximum number of iterations are set; Using the training set and the learning rate, the regularization parameter alpha, and the maximum number of iterations to train the emotion recognition model corresponding to each genetic individual, outputting the predicted discrete emotions, and obtaining the trained emotion recognition model corresponding to each genetic individual when the loss function reaches a threshold or the iteration reaches a maximum number of iterations; The test set is predicted using the trained emotion recognition model corresponding to each genetic individual, the accuracy is calculated based on the prediction result, and the inverse of the accuracy is used as the fitness value of each genetic individual.

4. The method for training an emotion recognition model according to claim 2, characterized in that: Each genetic individual has 4 eigenvalues, among which the first eigenvalue is the number of neurons in the first layer of the hidden layer, the second eigenvalue is the number of neurons in the second layer of the hidden layer, the third eigenvalue is the type of activation function, and the fourth eigenvalue is the type of optimizer.

5. The method for training an emotion recognition model according to any one of claims 2 to 4, characterized in that: If the termination condition is not met, a parental genetic individual pair is established based on the population, and the second eigenvalue and the fourth eigenvalue of the parental genetic individual pair are exchanged according to the crossover probability to generate a new genetic individual; the first eigenvalue or the second eigenvalue is randomly selected to generate a random number between [1,4], and the selected eigenvalue plus the random number is mutated as a new eigenvalue according to the mutation probability to generate a new genetic individual; Calculate the fitness value of the new genetic individual; combine the genetic individuals in the excellent population with the new genetic individuals, and select them according to the number of genetic individuals in the population and the fitness value of the genetic individuals in the combination to obtain an updated population.

6. The method for training an emotion recognition model according to claim 5, characterized in that: The establishing of the parental genetic individual pairs based on the population comprises: The first half of the genetic individuals in the sequence are selected to constitute the first parent generation, and the second half of the genetic individuals are selected to constitute the second parent generation; The genetic individuals in the first parent generation and the second parent generation form several parent generation genetic individual pairs in pairs according to the sequence numbers.

7. The method for training an emotion recognition model according to claim 5, characterized in that: All genetic individuals in the combination are sorted from small to large according to their fitness values, and the first N genetic individuals whose number is the same as that of the genetic individuals in the population are selected from the sorting to obtain an updated population.

8. The method for training an emotion recognition model according to claim 1, characterized in that: The emotions that appear in the videos watched by the subjects in the video material database are divided to construct a discrete emotion set; each discrete emotion in the discrete emotion set has high or low pleasure, high or low arousal attributes.

9. The method for training an emotion recognition model according to claim 8, characterized in that: After watching the currently selected video, the current subject makes an emotional evaluation on the video, that is, selects a corresponding discrete emotion from the discrete emotion set.

10. The method for training an emotion recognition model according to claim 9, characterized in that: The four-dimensional brain maps corresponding to all the subjects are annotated based on the emotion evaluation; wherein the annotated four-dimensional brain map corresponding to a subject watching a video is a sample, and the annotated four-dimensional brain maps corresponding to all the subjects constitute a data set.

Images