Emotion recognition and classification method based on neural network structure and group optimization fusion algorithm model

Abstract

The invention provides an emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model, and belongs to the technical field of electroencephalogram emotion recognition and classification. The method comprises the following steps: preprocessing electroencephalogram signals through downsampling, filtering and artifact removal; extracting local features and global features of the electroencephalogram signals by using a CDBN convolutional deep belief network; using a Bi-LSTM bidirectional long short-term memory network and a local attention mechanism to carry out feature enhancement on the extracted features; an ACO ant colony optimization algorithm is integrated, optimized features are used for model training, and an optimal model after training is saved; according to the emotion recognition and classification method and system, the accuracy of emotion recognition and classification is improved, and the training efficiency and performance of the model are improved.

Description

Technical Field

[0001] The present invention provides an emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model, and belongs to the technical field of electroencephalogram signal emotion recognition and classification. Background Art

[0002] EEG emotion recognition is a cutting-edge research field that uses electroencephalogram (EEG) signals to analyze and infer an individual's emotional state. Current research shows that different emotional states will present specific patterns in EEG signals. For example, pleasant emotions are usually accompanied by increased theta waves and decreased alpha waves in the left frontal region, while anxious emotions are associated with increased alpha waves in the right temporal region. In order to extract these emotion-related features, researchers have used various methods such as time-frequency domain, time-space domain, and complex network features.

[0003] In recent years, it has gradually become a trend to apply nonlinear analysis methods to the time-frequency domain and the spatial domain. Traditional EEG signal feature extraction algorithms mainly use methods such as convolutional neural networks (CNN), recurrent neural networks (RNN), support vector machines (SVM) and attention mechanisms. Although the above methods can achieve feature extraction of EEG signals, they still have problems such as large parameter requirements, high computational complexity and gradient disappearance, resulting in insufficient actual accuracy or long model training time.

[0004] Traditional machine learning methods such as support vector machines and random forests, as well as deep learning models such as convolutional neural networks and recurrent neural networks, have been widely used and have made significant progress. However, there are still many difficulties in using reliable EEG signals to robustly identify emotions. These difficulties are mainly caused by the nonlinear and non-stationary nature of EEG signals, such as high noise, weak signals, and individual differences in EEG signals. Traditional machine learning-based emotion recognition algorithms usually adopt a "feature extraction + classifier" structure. After manual feature extraction, vector machines, Bayesian networks and other algorithms are used for classification. Due to the need for manual feature extraction, they are constrained to a certain extent by the subjective experience of researchers; the model is sensitive to noise and cannot guarantee good robustness. Secondly, although there are many feature extraction algorithms for EEG signals, different EEG feature extraction methods will quantify EEG signals from different angles, so using a single feature extraction algorithm cannot give full play to the advantages of EEG signals. Summary of the invention

[0005] In view of the deficiencies of the above-mentioned technology, the purpose of the present invention is to provide an emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model. The method recognizes and classifies the patient's emotions through EEG signals based on the neural network structure and the group optimization fusion algorithm model.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] An emotion recognition and classification method based on a neural network structure and a swarm optimization fusion algorithm model is designed from the perspective of nonlinear analysis to extract EEG signal features and emotion classification. In the feature extraction process, the neural network structure uses a convolutional deep belief network, a bidirectional long short-term memory network and a local attention mechanism to reduce computational complexity and improve adaptability; the swarm optimization algorithm uses an ant colony optimization algorithm for parallelization and distributed optimization, improves global optimization capabilities, further reduces computational complexity and improves accuracy.

[0008] An emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model comprises the following steps:

[0009] (1) EEG data preprocessing;

[0010] (2) Extract EEG data features;

[0011] (3) Train the model and save the trained model;

[0012] (4) Use the saved model to perform EEG signal detection;

[0013] (5) Emotion recognition and classification.

[0014] It is further defined that the step (1) EEG data preprocessing specifically comprises:

[0015] ① Downsampling: The sampling frequency of the collected EEG signals is usually as high as 1000Hz. In order to reduce the amount of data and improve computing efficiency, the data set can be downsampled;

[0016] ② Filtering: Use Butterwort filter on the collected EEG signal. Since various acquisition devices may have low-frequency drift signals, the lower cutoff frequency of the filter is selected as 1Hz. However, there are various environmental noises and power frequency signal interferences in the experimental environment. These noises are all above 50Hz, so the upper cutoff frequency is selected as 50Hz.

[0017] ③ Artifact removal: Use independent component analysis to process the filtered EEG signal, and perform correlation analysis on each independent component with artifact signals such as electromyography and electrocardiography, so as to remove the artifact signals mixed in it.

[0018] It is further defined that step (2) extracts EEG data features, specifically comprising: using a convolutional deep belief network to process EEG data, the data is divided into two categories, one is EEG data in a calm state, and the other is EEG data that is stimulated and causes emotions, and these two types of data are input into the convolutional deep belief network to extract local features and global features.

[0019] It is further defined that the step (3) of training the model and saving the trained model specifically includes: using a bidirectional long short-term memory network for sequence modeling and feature learning; using a local attention mechanism for feature enhancement and weighting the importance of features output by the bidirectional long short-term memory network; using an ant colony optimization algorithm to optimize the hyperparameter selection of the model to improve the training efficiency and performance of the model, and then saving the trained model parameters as a state_dict format file.

[0020] It is further defined that the step (4) uses the saved model to perform EEG signal detection, specifically including: loading the state_dict file, loading the trained model parameters, inputting the EEG feature data to be classified and comparing it with the relevant features of the saved model, and identifying and outputting the possible emotion category contained in the EEG feature.

[0021] It is further defined that the step (5) emotion recognition and classification performs emotion classification through the fully connected layer and the Softmax layer of the network.

[0022] It is further defined that the downsampling reduces the brain wave sampling rate to 250 Hz to 500 Hz to reduce the amount of data.

[0023] It is further defined that the filtering has a passband frequency range of 0.5-45 Hz, a filter order of 4, a notch frequency of 50 / 60 Hz, and a notch quality factor of 30-50, which can effectively filter out power frequency interference and electromagnetic noise signals.

[0024] It is further defined that the artifact removal uses ICA independent component analysis, with amplitude threshold: mean ± 3σ, number of ICA components: equal to the number of electrode channels, segment length: 2 seconds, overlap rate: 50%, which effectively removes artifacts such as electromyography, electrooculography, and eye movement.

[0025] It is further defined that the EEG signal characteristics are mainly divided into three time-frequency domain characteristics: power spectral density, differential entropy, and sample entropy.

[0026] Further defined, the characteristic power spectrum density is specifically a physical quantity that characterizes the relationship between the power energy of a signal and its frequency. It is often used to study random vibration signals and is usually normalized according to the frequency resolution. First, the signal is converted to the frequency domain by Fourier transform. The Fourier transform of a continuous EEG signal X(t) with a length of T can be expressed as:

[0027] F(ω)=∫X(t)e -jωt dt

[0028] Where j is an imaginary number and ω is the frequency of the signal. However, the actual EEG signal collected is discrete, so for a discrete time signal X(n) with a length of N: {X(1), X(2)...X(N)}, n = 1, 2,..., N, its discrete Fourier transform is:

[0029]

[0030] Its power spectral density is:

[0031]

[0032] Further defined, the differential entropy feature is specifically a generalized form of Shannon information entropy in continuous variables, which is often used to count the total amount of information contained in continuous signals and quantify the total amount of uncertainty in the probability distribution of continuous random signals. Existing studies have shown that DE can effectively extract the logarithmic energy spectrum in fixed-length EEG signals, reflecting the complexity of fixed-length EEG signals. The original calculation formula of DE feature is:

[0033]

[0034] For an EEG signal T that obeys a Gaussian distribution N(μ,σ2), its DE can be defined as:

[0035]

[0036] Further defined, the sample entropy feature is specifically a time series complexity measurement method proposed by Richman, which can be represented by SampEn(m,r,N), where the dimensions are m and m+1, r is the similarity tolerance, and N is the length. Generally, for a time series consisting of N data {x(n)}=x(1),x(2)...x(N), the sample entropy is calculated as follows:

[0037]

[0038] It is further defined that the convolutional deep belief network uses a combination of a convolutional layer, a pooling layer and a deep belief network layer:

[0039] Convolution layer: convolution kernel size: 5×5, step size: 1, number of convolution kernels: 32, activation function: ReLU;

[0040] Pooling layer: Pooling type: max pooling, pooling window: 2×2, stride: 2.

[0041] It is further defined that the convolutional deep belief network, whose deep belief network includes multiple layers of restricted Boltzmann machines, the number of neurons in each layer is set to 256 or 512, sigmoid or tanh is selected as the activation function, and in order to prevent overfitting, Dropout is used in the convolutional layer, DBN layer and fully connected layer, usually set to between 0.2 and 0.5; the training batch size is usually selected to be 32 or 64, larger batches can speed up training, but smaller batches can sometimes bring better generalization ability; the number of training rounds is set to 50 to 100 rounds, and adjusted according to the convergence of the model.

[0042] It is further defined that the network layer parameters of the bidirectional long short-term memory network are:

[0043] Network layer parameters: input dimension: 64, hidden layer dimension: 128, number of LSTM layers: 2, bidirectional processing: True, sequence length: 62 (time step after downsampling);

[0044] Training parameters: batch size: 64, learning rate: 0.001, dropout rate: 0.3, recursive dropout rate: 0.3, gradient clipping threshold: 5.0;

[0045] Initialization strategy: Weight initialization: orthogonal initialization; Forget gate bias: initialized to 1.0; Other gate biases initialized to 0.

[0046] Further defining, the local attention mechanism, its parameter settings:

[0047] Structural parameters: input feature dimension: 256 (Bi-LSTM bidirectional output concatenation dimension), attention window size: 5, temperature parameter: 0.5, number of attention heads: 8;

[0048] Calculation parameters: key vector dimension: 32, query vector dimension: 32, value vector dimension: 32, attention dropout rate: 0.1;

[0049] Local context encoding: window filling mode: 'same', position encoding: relative position encoding, maximum relative distance: 10;

[0050] It is further defined that the optimization model of the ant colony optimization algorithm simulates the behavior of ants using pheromones and the local environment to make decisions in the process of searching for food. The algorithm assumes that the current best candidate solution position is the target prey position. After defining the target prey position, the ants randomly generate an initial solution in the solution space. The ants train the model according to the current parameter combination, set algorithm parameters such as the maximum number of iterations, colony size, etc., and calculate the performance of the model (such as the accuracy on the validation set, the loss function, etc.). According to the performance of the model, the parameter combination with better performance is selected, and pheromones are added to the corresponding path, and it is continuously iterated until the optimal parameter combination is found.

[0051] It is further defined that the algorithm model based on the fusion of neural network structure and swarm optimization is specifically: using a convolutional deep belief network to extract local features and global features based on the spatiotemporal domain, using a bidirectional long short-term memory network to capture the long-term dependencies of sequence features, combining a local attention module to highlight important timing features, and finally integrating the ant colony optimization algorithm to optimize the model hyperparameters and improve the reliability and efficiency of the model.

[0052] Further defined, the emotion recognition and classification uses the Plutchik emotion wheel model, which includes 8 basic emotions: joy, trust, fear, surprise, sadness, disgust, anger and expectation. The wheel model describes emotions according to intensity, with stronger emotions in the center and weaker emotions at the edge of the wheel. Like colors, basic emotions can be mixed to form complex emotions.

[0053] It is further defined that the EEG features are provided to a training model based on a neural network structure and group optimization fusion algorithm to obtain all possible emotions and their probabilities.

[0054] In the above technical scheme, the present invention provides an emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model, which has the following beneficial effects: the present invention combines a convolutional deep belief network, a bidirectional long short-term memory network, and local attention. Compared with traditional algorithms, the present invention can simultaneously capture local and global features of data at different levels, thereby improving the characterization ability and prediction accuracy of sequence data, reducing computational complexity, and improving sequence modeling capabilities. On this premise, it is integrated with the ant colony optimization algorithm to perform hyperparameter optimization and enhance feature quality, thereby improving the performance of the classifier, so that the classifier can more accurately identify and classify emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Reference numerals:

[0056] 1. Figure 1 A general framework flow chart of an emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model provided for the implementation of the present invention;

[0057] 2. Figure 2 A convolutional deep belief network structure framework diagram provided for the implementation of the present invention;

[0058] 3. Figure 3 A network structure framework diagram of a bidirectional long short-term memory network provided for the implementation of the present invention; DETAILED DESCRIPTION

[0059] As attached Figure 1 As shown, the overall process of the emotion recognition classification method based on the neural network structure and group optimization fusion algorithm model of the present invention is as follows:

[0060] 1. First, at the beginning of the experiment, the person to be tested needs to sit quietly and rest. When the heart rate of the person to be tested tends to stabilize the resting heart rate, the brain wave data of the person to be tested in a calm state is collected as a reference signal.

[0061] 2. After collecting EEG data in a calm state, use different power segments to stimulate the person to be tested to produce different emotional changes, and collect EEG data containing different emotions.

[0062] 3. Downsample the collected EEG signal data, reduce the sampling rate to 250Hz, and then perform noise reduction processing such as independent component analysis, wavelet transform, filtering, etc. to remove power frequency interference, environmental noise, electrooculography, and electromyography artifacts. Among them, the wavelet transform recursively decomposes the signal by decomposing it into low-frequency and high-frequency components. Its mathematical expression is:

[0063]

[0064] Among them, c jk is the wavelet coefficient, ψ jk ( t ) represents the characteristics of the signal at different scales and positions. By iteratively decomposing the high-frequency and low-frequency parts of the signal, a multi-resolution representation of the signal at different scales can be obtained.

[0065] 4. The preprocessed EEG signal is extracted using a convolutional deep belief network, and the convolutional layer formula is:

[0066] y=(x*w)+b

[0067] Where x is the input signal (which can be a time window or frequency domain representation of EEG), w is the convolution kernel (or filter), b is the bias term, and y is the output feature map after convolution.

[0068] Then, the multi-layer restricted Boltzmann machine (CRBM) is used for adversarial learning. At the same time, the extracted features are further strengthened using a bidirectional long short-term memory network. Model long-term and short-term dependencies, and then use the local attention mechanism to weight the features. The formula is:

[0069]

[0070] Emphasizing the signal parts that have important influence on emotions, the ant colony optimization algorithm optimizes the hyperparameters of the model through local and global updates.

[0071] 5. After the model training is completed, save the model parameters, use the trained model to perform new emotion recognition classification, and output the emotion that the EEG data is most likely to match.

[0072] 6. The results of the method of the present invention are compared with those of three models: convolutional neural network, bidirectional long short-term memory network, and convolutional neural network + long short-term memory network + local attention mechanism. The results show that the present invention not only further improves the accuracy of the traditional algorithm model, but also significantly improves the training rate.

[0073] The method provided by the present invention recognizes and classifies different emotions of the subject based on EEG signals through a neural network structure and a group optimization fusion algorithm model.

Claims

1. An emotion recognition and classification method based on a neural network structure and a group optimization fusion algorithm model, characterized in that: The method specifically comprises: (1) EEG data preprocessing; (2) Extract EEG data features; (3) Train the model and save the trained model; (4) Use the saved model to detect EEG signals; (5) Emotion recognition and classification.

2. The emotion recognition and classification method based on the neural network structure and group optimization fusion algorithm model according to claim 1 is characterized in that: The (1) EEG data preprocessing specifically includes: ① downsampling ② filtering ③ artifact removal.

3. The emotion recognition and classification method based on the neural network structure and group optimization fusion algorithm model according to claim 1 is characterized in that: The (2) extracting EEG data features specifically includes: using a convolutional deep belief network to process EEG data, where the data is divided into two categories, one is EEG data in a calm state, and the other is EEG data that is stimulated and causes emotions. These two categories of data are input into the convolutional deep belief network to extract local features and global features.

4. The emotion recognition and classification method based on the neural network structure and group optimization fusion algorithm model according to claim 1 is characterized in that: The (3) training model and saving the trained model specifically include: using a bidirectional long short-term memory network, a local attention mechanism and an ant colony optimization algorithm to form a fusion algorithm model, performing enhanced feature training, and saving model parameters after training.

5. The emotion recognition and classification method based on the neural network structure and group optimization fusion algorithm model according to claim 1 is characterized in that: The (4) method uses the saved model to detect EEG signals, specifically including: training a fusion algorithm model of time domain and frequency domain, comparing the EEG feature data to be classified with the relevant features of the saved model after training, and identifying and outputting the possible emotion categories contained in the EEG feature.

6. The emotion recognition and classification method based on the neural network structure and group optimization fusion algorithm model according to claim 1 is characterized in that: The step (5) emotion recognition and classification is to classify emotions through the fully connected layer and the Softmax layer of the model.

Images