Pilot emotion classification method based on EEG signals

Abstract

The invention provides an EEG signal-based pilot emotion classification method, which comprises the following steps of: acquiring EEG signals of a pilot in a flight simulation task, carrying out preprocessing operations such as adaptive filtering, data standardization and segmentation on the EEG signals, extracting various characteristics such as power spectral density, differential entropy, coherence and phase synchronization, and fusing, so as to obtain an emotion classification result of the pilot. And constructing a recursive polynomial network comprising an input layer, a recursive polynomial layer and an output layer. The output of a polynomial unit in the recursive polynomial layer is related to the state of the previous moment, and the output is calculated and the state is updated through a specific formula. The model is trained by adopting a cross entropy loss function and an Adam optimization algorithm, evaluation and optimization are carried out according to indexes such as accuracy and recall rate, and finally pilot emotion classification is realized. The method can comprehensively and accurately recognize the emotional state of the pilot, and has important application value in the aspects of flight safety monitoring, task allocation, mental health management and the like.

Description

Technical Field

[0001] The present invention belongs to the technical field of emotion classification, and particularly relates to a method for classifying pilots' emotions based on EEG signals. Background Art

[0002] In the field of aviation flight, the emotional state of pilots is crucial for flight safety. Emotional changes can affect pilots' cognitive abilities, decision-making abilities, and operation accuracy, thereby affecting the execution of flight tasks and flight safety. Therefore, it is of great significance to accurately identify the emotional state of pilots in real time. Currently, in the field of emotion recognition, various methods have been applied. Among them, the emotion recognition method based on physiological signals has received extensive attention due to its objectivity and directness. Electroencephalogram (EEG), as an important physiological signal, can directly reflect the neural activities inside the brain and has unique advantages in emotion recognition. However, EEG signals have characteristics such as complexity, non-linearity, and high dimensionality, and traditional emotion classification methods have certain limitations in processing EEG signals. When traditional neural networks process high-degree polynomials, they often face problems such as overfitting and gradient vanishing, resulting in poor generalization ability and unstable training of the model. Although conventional algorithms such as neural networks have solved these problems to a certain extent, there is still room for improvement. Summary of the Invention

[0003] In view of this, the present invention aims to overcome the above deficiencies in the prior art and proposes a method for classifying pilots' emotions based on EEG signals.

[0004] To achieve the above object, the technical solution of the present invention is realized as follows:

[0005] The first aspect of the present invention provides a method for classifying pilots' emotions based on EEG signals, including the following steps:

[0006] Step 1: Obtain EEG signal data and preprocess the EEG signals.

[0007] Step 2: Perform band decomposition on the EEG signals based on a frequency band width of 2 Hz, decompose the EEG signals into multiple sub-bands, extract the power spectral density features, differential entropy features, coherence features, and phase synchrony features on each sub-band, and perform feature fusion to obtain a feature sequence.

[0008] Step 3: Construct a recursive polynomial network classification model, including an input layer, a recursive polynomial layer, and an output layer, and use the Adam optimization algorithm to optimize the parameters of the recursive polynomial network classification model.

[0009] Step 4: Construct a training set and a test set, train the recursive polynomial network classification model based on the training set, and verify the classification performance of the model on the test set.

[0010] Step 5: Input the feature sequence obtained in Step 2 into the recursive polynomial network classification model to obtain the classification result.

[0011] Further, in Step 1, a multi-channel physiological recorder is used to collect the EEG signals of the pilot during the flight simulation task. The sampling frequency is 500 Hz, and the electrode placement adopts the "10-20" international lead standard. A total of 34 channels of EEG signals are collected. The preprocessing of the EEG signals includes denoising, data normalization, and data segmentation.

[0012] Further, in Step 2,

[0013] The calculation formula for the power spectral density feature is:

[0014]

[0015] where f is the frequency, s(t) is the EEG signal, T is the number of data points, and n is the index of the frequency, representing different frequency components;

[0016] The calculation formula for the differential entropy feature is:

[0017] DE = -∫p(x)ln p(x)dx;

[0018] where p(x) is the probability density function of the signal;

[0019] The calculation formula for the coherence feature is:

[0020]

[0021] where S IJ (f) is the cross-spectral density of channel I and channel J at frequency f, and S II (f) and S JJ (f) are the auto-spectral densities of channel I and channel J at frequency f, respectively;

[0022] The calculation formula for the phase synchronization feature is:

[0023]

[0024] where and are the phases of channel I and channel J at time t, respectively.

[0025] Further, in step 3, the input layer of the recursive polynomial network classification model is used to receive the fused feature vector. The recursive polynomial layer consists of multiple polynomial units. The output of each polynomial unit depends on the current input and the state at the previous moment. The output layer performs classification through the softmax function and outputs the probability of the emotion category.

[0026] Further, in step 3, the calculation process of the polynomial unit is as follows:

[0027] Let the input vector be x(t) = [x1(t), x2(t),..., x n (t)] T , and the state at the previous moment be s(t - 1) = [s1(t - 1), s2(t - 1),..., s n (t - 1)] T , and the output of the polynomial unit be y(t) = [y1(t), y2(t),..., y p (t)] T , Q(i, j) represents the set of each power term. The expression of Q(i, j) can be obtained as follows:

[0028]

[0029] Among them, σ q，i is the power of the variable x i (t) in the q-th product term of this item. Here, i represents the starting subscript of the power term of x, and j represents the starting subscript of the power term of s. s(t) is represented by multiplying the power term by the corresponding weight value and then summing:

[0030]

[0031] Among them, is the weight coefficient of the polynomial unit;

[0032] s(t) enters the softmax layer to obtain the model prediction.

[0033] Further, in step 4, when training the model, the loss function used is the cross-entropy loss function, and the formula is:

[0034]

[0035] Among them, N is the number of samples, C is the number of emotion categories, y c (n) is the true category label of the n-th sample in category c, is the probability that the model prediction result belongs to category c.

[0036] Further, in the step 4, the Adam optimization algorithm is used to optimize the parameters of the recursive polynomial network classification model. The update formula of the Adam algorithm is as follows:

[0037] m k = β1m k-1 + (1 - β1)g k ;

[0038]

[0039] where w k-1 is the weight of the previous iteration; w k is the weight obtained in this iteration, k is the number of iterations; m k and v k are the first-order moment estimate and the second-order moment estimate of the gradient at the k-th iteration respectively, and are the corrected first-order moment estimate and the second-order moment estimate at the k-th iteration, β1 and β2 are the exponential decay rates, α is the learning rate, ∈ is a small constant to prevent division by zero, and g k is the gradient at the k-th iteration.

[0040] The second aspect of the present invention provides a pilot emotion classification device based on EEG signals, including:

[0041] A data processing unit, configured to acquire EEG signal data and preprocess the EEG signals;

[0042] A feature extraction unit, configured to perform band decomposition on the EEG signals based on a frequency band width of 2 Hz, decompose the EEG signals into multiple sub-frequency bands, extract the power spectral density features, differential entropy features, coherence features, and phase synchronization features on each sub-frequency band, and perform feature fusion to obtain a feature sequence;

[0043] A model construction unit, configured to construct a recursive polynomial network classification model, including an input layer, a recursive polynomial layer, and an output layer, and optimize the parameters of the recursive polynomial network classification model by using the Adam optimization algorithm;

[0044] A model training unit, configured to construct a training set and a test set, train the recursive polynomial network classification model based on the training set, and verify the classification performance of the model on the test set;

[0045] A result output unit, configured to input the feature sequence obtained by the feature extraction unit into the recursive polynomial network classification model to obtain a classification result.

[0046] A third aspect of the present invention provides an electronic device, including a processor and a memory communicatively connected to the processor and used for storing executable instructions of the processor, and the processor is used for executing the above-mentioned method for classifying pilot emotions based on EEG signals.

[0047] A fourth aspect of the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the above-mentioned method for classifying pilot emotions based on EEG signals is implemented.

[0048] Compared with the prior art, the method for classifying pilot emotions based on EEG signals of the present invention has the following advantages:

[0049] 1) By collecting EEG signals of pilots and extracting various features, the present invention can comprehensively and accurately identify the emotional states of pilots, which helps to monitor the emotional changes of pilots in real time, improve flight safety and mission execution efficiency;

[0050] 2) The constructed recursive polynomial network classification model can effectively process the long-term dependence relationships in time series data, improve the accuracy and stability of emotion classification, and has better generalization ability and training stability compared with traditional neural network models;

[0051] 3) Using the Adam optimization algorithm to optimize the model parameters can dynamically adjust the learning rate, improve the training efficiency and convergence speed of the model, ensure that the model can find the global optimal or local optimal solution, and improve the classification accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] The drawings constituting a part of the present invention are used to provide a further understanding of the present invention. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0053] Figure 1 is a framework diagram of the method for classifying pilot emotions based on EEG signals of the present invention;

[0054] Figure 2 is a structural schematic diagram of the recursive polynomial network classification model of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0055] It should be noted that, without conflict, the embodiments in the present invention and the features in the embodiments can be combined with each other.

[0056] In the description of the present invention, it should be understood that the orientation or positional relationships indicated by the terms "center", "longitudinal", "transverse", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. are based on the orientation or positional relationships shown in the drawings. These are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation of the present invention. In addition, terms such as "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, features defined with "first", "second", etc. may explicitly or implicitly include one or more of such features. In the description of the present invention, unless otherwise specified, the meaning of "a plurality" is two or more.

[0057] In the description of the present invention, it should be noted that unless otherwise clearly defined and limited, the terms "mounted", "connected", and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood through specific circumstances.

[0058] The present invention will be described in detail below with reference to the drawings and in conjunction with embodiments.

[0059] Embodiment 1:

[0060] As Figure 1 shown, the present invention provides a method for classifying pilot emotions based on EEG signals. It starts from the data acquisition link, and obtains the EEG signals of pilots during flight simulation tasks at a specific sampling frequency and lead standard. Then it enters the preprocessing and frequency band decomposition stage, using adaptive filtering technology to denoise, standardize the signal amplitude and segment the signal according to time windows. Subsequently, feature extraction is carried out, covering various features such as power spectral density, differential entropy, coherence, and phase synchrony, and they are fused. Finally, a Recursive Polynomial Network (RPN) classification model is constructed. The feature vector is received through the input layer, processed by the recursive polynomial layer, and the probability of the emotion category is output by the output layer to complete the entire pilot emotion classification process. Specifically, it includes the following steps:

[0061] Step 1, obtain EEG signal data and preprocess the signal;

[0062] Data acquisition: A multi-channel physiological recorder was used to collect the EEG signals of pilots during flight simulation tasks. The sampling frequency was 500 Hz, and the electrode placement followed the "10-20" international lead standard. A total of 34 channels of EEG signals were collected.

[0063] Data preprocessing: Noise was removed. That is, the adaptive filtering technique was used to remove noise from the collected EEG signals. The formula for the adaptive filtering algorithm is:

[0064]

[0065] where t represents discrete time points, z′(t) is the filtered output signal, z(t) is the input signal, w i (t) is the weight coefficient of the adaptive filter, and M is the filter order.

[0066] Data normalization: The signal amplitude was mapped to the interval [-1, 1]. The normalization formula is:

[0067]

[0068] where s(t) is the normalized EEG signal, μ is the signal mean, and σ is the signal standard deviation.

[0069] Data segmentation: The preprocessed EEG signals were segmented with a time window length of 1 second and an overlap rate of 25% to obtain a series of EEG data segments.

[0070] Step 2: Based on a frequency band width of 2 Hz, the EEG signals were decomposed into multiple sub-frequency bands. The power spectral density features, differential entropy features, coherence features, and phase synchronization features of each sub-frequency band were extracted, and feature fusion was performed to obtain a feature sequence. Among them:

[0071] Power spectral density (PSD) features: The fast Fourier transform (FFT) was performed on the EEG signal data segments of each channel, and the power spectral density of different frequency bands (such as α wave: 8 - 13 Hz, β wave: 14 - 30 Hz, θ wave: 4 - 7 Hz, δ wave: 0.5 - 3 Hz) was calculated. The PSD calculation formula is:

[0072]

[0073] where f is the frequency, s(t) is the normalized EEG signal, T is the number of data points, and n is the index of the frequency, representing different frequency components.

[0074] Differential entropy (DE) features: The differential entropy of the EEG signal data segments was calculated. The DE calculation formula is:

[0075] DE = -∫p(x)ln p(x)dx; where p(x) is the probability density function of the signal.

[0076] Coherence feature: Calculate the coherence between EEG signals of different channels. The coherence calculation formula is:

[0077]

[0078] where IJ S II (f) is the cross - spectral density between channel I and channel J at frequency f, and S JJ (f) and S

[0079] Phase Synchronization feature: Calculate the phase synchronization between EEG signals of different channels. The phase synchronization calculation formula is:

[0080]

[0081] where and are the phases of channel I and channel J at time t respectively.

[0082] Feature fusion: Fuse the above - extracted PSD, DE, Coherence, and Phase Synchronization features to obtain the fused feature vector x(t), t = 1, … T.

[0083] Step 3: Construct a Recursive Polynomial Network (RPN) classification model. The model consists of an input layer, a recursive polynomial layer, and an output layer. The input layer receives the EEG signal feature sequence X after band - decomposition. The middle layer approximates the non - linear relationship between the input and output through a polynomial expansion with the highest power of m. The output layer combines a recurrent connection mechanism to finally output the classification result Y of the model. The recurrent connection mechanism can handle the temporal dependence relationship in sequence data. The hidden state at each time step is passed to the next time step through the recurrent connection and participates in the calculation as part of the input to capture the long - term dependence information in the sequence.

[0084] The structure of the constructed Recursive Polynomial Network (RPN) classification model is as Figure 2 shown:

[0085] Recursive Polynomial Network Structure: Construct a recursive polynomial network, including an input layer, a recursive polynomial layer, and an output layer. The input layer is used to receive the fused feature vectors. The recursive polynomial layer consists of multiple polynomial units. The output of each polynomial unit depends not only on the current input but also on the state at the previous moment. The output layer performs classification through the softmax function and outputs the probability of the emotion category.

[0086] Calculation of Polynomial Unit: The calculation process of the polynomial unit is as follows. Let the input vector be x(t) = [x1(t), x2(t),..., x n (t)] T , and the state at the previous moment, that is, the output composed of polynomials, is s(t - 1) = [s1(t - 1), s2(t - 1),..., s n (t - 1)] T The output of the polynomial unit is y(t) = [y1(t), y2(t),..., y p (t)] T , and Q(i, j) represents the set of each power term. The expression of Q(i, j) can be obtained as follows:

[0087]

[0088] Among them, σ q，i is the power of the variable x i (t) in the qth product term of this item, where i represents the starting subscript of the power terms of x, and j represents the starting subscript of the power terms of s. Then s(t) can be expressed as the sum of the products of the power terms and the corresponding weights, that is

[0089]

[0090] Among them, is the weight coefficient of the polynomial unit. As shown above, the output of the polynomial, that is, the state s, has the same dimension as x, and s(t - 1) is fed back to the input end and input into the polynomial layer together with x(t). s(t) enters the softmax layer to obtain the model prediction

[0091] Step 4: Use the Adam optimization algorithm to adaptively adjust the parameters of the RPN model. Combine the learning rate decay coefficient to dynamically adjust the learning rate of each parameter to ensure that the model can find the global optimal or local optimal solution and improve the classification accuracy;

[0092] The update formula of the Adam algorithm is:

[0093] m k = β1m k-1 +(1 - β1)g k ;

[0094]

[0095] Among them, w k-1 is the weight of the previous iteration; w k is the weight obtained in this iteration, and k is the number of iterations; m k and v k are the first-order moment estimate and the second-order moment estimate of the gradient at the k-th iteration respectively, and are the corrected first-order moment estimate and the second-order moment estimate at the k-th iteration, β1 and β2 are the exponential decay rates, α is the learning rate, ∈ is a small constant to prevent division by zero, and g k is the gradient at the k-th iteration.

[0096] Step Five: Train the RPN model based on the training set, and in combination with the publicly available emotion dataset (such as the DEAP dataset, etc.), which contains EEG signals and corresponding emotion labels of multiple subjects under different emotion induction conditions. Randomly divide the dataset into a training set and a test set according to the ratio of 70%:30%, ensuring that the training set and the test set are consistent in the distribution of emotion categories. Verify the classification performance of the model on the test set to obtain the classification results of the pilot's emotions.

[0097] When training the model, use the cross-entropy loss function, and the formula is:

[0098]

[0099] Among them, N is the total number of samples, C is the total number of emotion categories, y c (n) is the true category label of the n-th sample in category c, is the probability that the model prediction result belongs to category c.

[0100] Example Two:

[0101] A pilot emotion classification device based on EEG signals, comprising:

[0102] A data processing unit, configured to acquire EEG signal data and preprocess the EEG signals;

[0103] A feature extraction unit, configured to perform band decomposition on the EEG signals based on a frequency band width of 2 Hz, decompose the EEG signals into multiple sub-frequency bands, extract power spectral density features, differential entropy features, coherence features, and phase synchronization features on each sub-frequency band, and perform feature fusion to obtain a feature sequence;

[0104] A model construction unit for constructing a recursive polynomial network classification model, including an input layer, a recursive polynomial layer, and an output layer, and using the Adam optimization algorithm to optimize the parameters of the recursive polynomial network classification model;

[0105] A model training unit for constructing a training set and a test set, training the recursive polynomial network classification model based on the training set, and verifying the classification performance of the model on the test set;

[0106] A result output unit for inputting the feature sequence obtained by the feature extraction unit into the recursive polynomial network classification model to obtain a classification result.

[0107] Embodiment III:

[0108] An electronic device includes a processor and a memory communicatively connected to the processor and used for storing executable instructions of the processor, and the processor is used to execute the above-mentioned method for classifying pilot emotions based on EEG signals.

[0109] Embodiment IV:

[0110] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the above-mentioned method for classifying pilot emotions based on EEG signals.

[0111] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A pilot emotion classification method based on EEG signals, characterized by: The steps include: Step 1: Obtain EEG signal data and preprocess the EEG signal; Step 2: Based on the frequency bandwidth of 2 Hz, the EEG signal is decomposed into multiple sub-bands, and the power spectrum density features, differential entropy features, coherence features and phase synchronization features on each sub-band are extracted, and feature fusion is performed to obtain a feature sequence; Step 3: Construct a recursive polynomial network classification model, including an input layer, a recursive polynomial layer, and an output layer, and use the Adam optimization algorithm to optimize the parameters of the recursive polynomial network classification model; Step 4: Construct a training set and a test set, train the recursive polynomial network classification model based on the training set, and verify the classification performance of the model on the test set; Step 5: Input the feature sequence obtained in step 2 into the recursive polynomial network classification model to obtain the classification result.

2. The method for pilot emotion classification based on EEG signals according to claim 1, characterized in that: In step 1, a multi-channel physiological recorder is used to collect EEG signals of pilots in flight simulation tasks, the sampling frequency is 500 Hz, the electrode placement adopts the "10-20" international lead standard, and a total of 34 channels of EEG signals are collected. The EEG signals are preprocessed, including denoising, data standardization, and data segmentation.

3. The method for pilot emotion classification based on EEG signals according to claim 1, characterized in that: In the step 2, The power spectrum density characteristic calculation formula is: Where f is the frequency, s(t) is the EEG signal, T is the number of data points, and n is the frequency index, representing different frequency components; The differential entropy feature calculation formula is: DE = -∫p(x)ln p(x)dx; Where p(x) is the probability density function of the signal; The formula for calculating the coherence characteristic is: Among them, S IJ (f) is the cross-spectral density of channel I and channel J at frequency f, S II (f) and S JJ (f) are the autospectral densities of channel I and channel J at frequency f, respectively; The phase synchronization characteristic calculation formula is: in, and are the phases of channel I and channel J at time t respectively.

4. The method for pilot emotion classification based on EEG signals according to claim 1, characterized in that: In step 3, the input layer of the recursive polynomial network classification model is used to receive the fused feature vector, the recursive polynomial layer is composed of multiple polynomial units, the output of each polynomial unit depends on the current input and the state at the previous moment, and the output layer is classified by the softmax function to output the emotion category probability.

5. The method for pilot emotion classification based on EEG signals according to claim 4, characterized in that: In step 3, the calculation process of the polynomial unit is as follows: Let the input vector be x(t)=[x1(t), x2(t), ..., x n (t)] T , the state at the previous moment is s(t-1)=[s1(t-1), s2(t-1), ..., s n (t-1)] T , the output of the polynomial unit is y(t)=[y1(t),y2(t),...,y p (t)] T , Q(i, j) represents the set of power terms, and the expression of Q(i, j) is as follows: Among them, σ q，i is the variable x in the qth product term in the corresponding term i (t), where i represents the starting subscript of the power of x, and j represents the starting subscript of the power of s. The power term is multiplied by the corresponding weight and then summed to represent s(t): in, is the weight coefficient of the polynomial unit; s(t) enters the softmax layer to get the model prediction.

6. The pilot emotion classification method based on EEC signal according to claim 1, characterized in that: In step 4, when training the model, the loss function used is the cross entropy loss function, and the formula is: Among them, N is the number of samples, C is the number of emotion categories, and y c (n) is the true category label of the nth sample in category c, The probability that the model predicts that the result belongs to category c.

7. The method for pilot emotion classification based on EEG signals according to claim 1, characterized in that: In step 4, the Adam optimization algorithm is used to optimize the parameters of the recursive polynomial network classification model. The update formula of the Adam optimization algorithm is: m k =β1m k-1 +(1-β1)g k ; Among them, w k-1 is the weight of the previous iteration; w k is the weight obtained in this iteration, k is the number of iterations; m k and v k are the first-order moment estimate and the second-order moment estimate of the gradient at the kth iteration, respectively. and is the modified first-order moment estimate and second-order moment estimate at the kth iteration, β1 and β2 are exponential decay rates, α is the learning rate, ∈ is a small constant to prevent division by zero, and g k is the gradient at the kth iteration.

8. The pilot emotion classification device based on EEG signals according to claim 1, characterized in that: include: A data processing unit, used for acquiring EEG signal data and preprocessing the EEG signal; A feature extraction unit is used to perform frequency band decomposition on the EEG signal based on a frequency band width of 2 Hz, decompose the EEG signal into multiple sub-bands, extract the power spectrum density feature, differential entropy feature, coherence feature and phase synchronization feature on each sub-band, and perform feature fusion to obtain a feature sequence; A model building unit is used to build a recursive polynomial network classification model, including an input layer, a recursive polynomial layer and an output layer, and uses the Adam optimization algorithm to optimize the parameters of the recursive polynomial network classification model; A model training unit is used to construct a training set and a test set, train the recursive polynomial network classification model based on the training set, and verify the classification performance of the model on the test set; The result output unit is used to input the feature sequence obtained by the feature extraction unit into the recursive polynomial network classification model to obtain the classification result.

9. An electronic device, comprising a processor and a memory connected to the processor for storing instructions executable by the processor, characterized in that: The processor is used to execute the pilot emotion classification method based on EEG signals as described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for classifying pilot emotions based on EEG signals as described in any one of claims 1 to 7 is implemented.

Images