Emotional state recognition method and device based on electroencephalogram signals and electronic equipment

Abstract

The invention provides an emotional state recognition method and device based on electroencephalogram signals and electronic equipment. Firstly, a target electroencephalogram signal of a to-be-analyzed object is obtained; then, feature extraction is conducted on the target electroencephalogram signal, a delta-alpha frequency band power ratio set, a self-adaptive similar tolerance fuzzy entropy set and a phase lag index set are obtained, and each element in the delta-alpha frequency band power ratio set represents the collaboration degree between the activity of a delta frequency band and the activity of an alpha frequency band in the target electroencephalogram signal; each element in the adaptive similar tolerance fuzzy entropy set represents the complexity of the target electroencephalogram signals, and each element in the phase lag index set represents the phase synchronization degree of the target electroencephalogram signals of the two channels. And then fusing the delta-alpha frequency band power ratio set, the adaptive similar tolerance fuzzy entropy set and the phase lag index set to obtain a fused feature set. And finally, based on the fusion feature set, performing emotional state recognition on the to-be-analyzed object to obtain an emotional state recognition result.

Description

Technical Field

[0001] The present application relates to the field of electroencephalogram technology, and particularly to a method, device and electronic device for identifying an emotional state based on electroencephalogram signals. Background Art

[0002] Since electroencephalogram (EEG) signals can reflect subtle changes in the brain nerve activities of a detected object, they are widely applied to the research on emotional state recognition. Currently, there is a problem of low accuracy in recognizing an emotional state by using electroencephalogram signal technology. Summary of the Invention

[0003] The main purpose of the embodiments of the present application is to propose a method, device and electronic device for identifying an emotional state based on electroencephalogram signals, aiming to improve the accuracy of identifying an emotional state based on electroencephalogram signals.

[0004] To achieve the above object, a first aspect of the embodiments of the present application proposes a method for identifying an emotional state based on electroencephalogram signals, the method including:

[0005] Obtaining target electroencephalogram signals of multiple channels of an object to be analyzed;

[0006] Performing feature extraction on the target electroencephalogram signals of the multiple channels to obtain a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set, where each element in the δ-α band power ratio set represents the cooperation degree between the activities of the δ band and the α band in the target electroencephalogram signals, each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target electroencephalogram signals, and each element in the phase lag index set represents the phase synchronization degree of the target electroencephalogram signals of two channels;

[0007] Fusing the δ-α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set to obtain a fused feature set;

[0008] Based on the fused feature set, performing emotional state recognition on the object to be analyzed to obtain an emotional state recognition result of the object to be analyzed.

[0009] A second aspect of the embodiments of the present application proposes a device for identifying an emotional state based on electroencephalogram signals, including:

[0010] An obtaining module, configured to obtain target electroencephalogram signals of multiple channels of an object to be analyzed;

[0011] A feature extraction module, configured to extract features from the target electroencephalogram (EEG) signals of multiple channels, so as to obtain a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set. Each element in the δ-α band power ratio set represents the degree of cooperation between the activities of the δ band and the α band in the target EEG signal. Each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target EEG signal. Each element in the phase lag index set represents the degree of phase synchronization of the target EEG signals of two channels;

[0012] A feature fusion module, configured to fuse the δ-α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set to obtain a fused feature set;

[0013] An identification result acquisition module, configured to perform an emotional state identification on the object to be analyzed based on the fused feature set, so as to obtain an emotional state identification result of the object to be analyzed.

[0014] A third aspect of the embodiments of the present application provides an electronic device, which includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, the above-mentioned method is implemented.

[0015] In the method for identifying an emotional state based on EEG signals according to some embodiments of the present application, first, the target EEG signals of the object to be analyzed are obtained. Then, features are extracted from the target EEG signals to obtain a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set. Each element in the δ-α band power ratio set represents the degree of cooperation between the activities of the δ band and the α band in the target EEG signal. Each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target EEG signal. Each element in the phase lag index set represents the degree of phase synchronization of the target EEG signals of two channels. After that, the δ-α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set are fused to obtain a fused feature set. Finally, an emotional state identification is performed on the object to be analyzed based on the fused feature set to obtain an emotional state identification result. In this way, the fused feature set characterizes the target EEG signals of the object to be analyzed more accurately from three dimensions that have a strong correlation with the emotional state, namely, the degree of cooperation between the activities of the δ band and the α band, the complexity of the target EEG signal, and the degree of phase synchronization of the target EEG signals of two channels, which is beneficial to improving the accuracy of the emotional state identification result of the object to be analyzed. Description of the Drawings

[0016] Figure 1A schematic flowchart of a method for recognizing an emotional state based on electroencephalogram (EEG) signals provided by an embodiment of the present application;

[0017] Figure 2 Provided by an embodiment of the present application Figure 1 A schematic flowchart of step S200 shown;

[0018] Figure 3 Provided by an embodiment of the present application Figure 2 A schematic flowchart of step S220 shown;

[0019] Figure 4 Provided by an embodiment of the present application Figure 1 Another schematic flowchart of step S200 shown;

[0020] Figure 5 Provided by an embodiment of the present application Figure 1 A schematic flowchart of step S400 shown;

[0021] Figure 6 Provided by an embodiment of the present application Figure 5 A schematic flowchart of step S410 shown;

[0022] Figure 7 A schematic structural diagram of a device for recognizing an emotional state based on EEG signals provided by an embodiment of the present application;

[0023] Figure 8 A schematic hardware structure diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present application without creative efforts shall fall within the protection scope of the present application.

[0025] An embodiment of the present application provides a method, a device, and an electronic device for recognizing an emotional state based on EEG signals, aiming to improve the accuracy of recognizing an emotional state based on EEG signals.

[0026] Figure 1 A schematic flowchart of the method for recognizing an emotional state provided by an embodiment of the present application. Figure 1 The method shown includes steps S100 to S400.

[0027] Step S100, obtaining target EEG signals of multiple channels of an object to be analyzed.

[0028] In step S100, target EEG signals of multiple channels of the object to be analyzed are obtained, which can provide a rich data source for subsequent comprehensive feature extraction of the EEG signals, ensure that the EEG activities of the object to be analyzed can be analyzed from multiple angles, and thus more accurately identify the emotional state.

[0029] In some embodiments, obtaining the target EEG signals of multiple channels of the object to be analyzed includes: obtaining the raw EEG signals of multiple channels of the object to be analyzed, and preprocessing the raw EEG signals of multiple channels to obtain the target EEG signals of multiple channels of the object to be analyzed.

[0030] In some embodiments, the emotional state of the object to be analyzed may include but is not limited to abnormal emotional states and normal emotional states. Abnormal emotional states include but are not limited to anxiety and depression. Normal emotional states include but are not limited to calmness and excitement.

[0031] In some embodiments, the raw EEG signals of multiple channels can be obtained from an EEG acquisition device. Among them, the EEG acquisition device usually includes multiple electrodes, and these electrodes can be placed at specific positions on the scalp of the object to be analyzed. Each electrode can collect the EEG signal of a brain region and output the electrode signal of one channel. Therefore, the EEG acquisition device can simultaneously obtain the electrode signals of multiple channels collected by multiple electrodes, and then obtain the raw EEG signals of multiple channels based on the electrode signals of multiple channels. For example, when the number of electrodes of the EEG acquisition device is 128, the raw EEG signals with a dimension of 128 channels can be collected.

[0032] In some embodiments, the preprocessing includes one or more of denoising, filtering, artifact removal, abnormal channel deletion, abnormal data point deletion, bad channel interpolation, and rereferencing, so as to improve the overall quality of the target EEG signals of multiple channels, and further improve the accuracy of feature extraction and recognition of the target EEG signals.

[0033] In some embodiments, the raw EEG signals can be denoised and filtered to remove the noise in the raw EEG signals and retain the signals within a preset frequency range to obtain clearer and more useful signals. In some embodiments, the preset frequency can be 1 Hz to 40 Hz.

[0034] In some embodiments, artifact removal can also be performed on the raw EEG signals. Artifact removal includes at least one of motion artifact removal and electrooculogram artifact removal to remove the interference signals from muscle movement or eye movement.

[0035] In some embodiments, a pre-set reference average value and standard deviation value may also be used to delete abnormal channels and abnormal data points from the original EEG signals, so as to improve the overall quality of the target EEG signals of multiple channels and ensure the accuracy of subsequent analysis of the target EEG signals.

[0036] In some embodiments, bad channel interpolation may also be performed on the original EEG signals, aiming to restore and maintain the integrity and consistency of the original EEG signals and ensure the accuracy and reliability of subsequent analysis of the target EEG signals. Among them, the methods of bad channel interpolation include but are not limited to spherical interpolation.

[0037] In some embodiments, a whole-brain average reference may also be used to re-reference the original EEG signals. Compared with using techniques such as reference electrode standardization for re-referencing, the whole-brain average reference can reduce the deviation effect of monopolar reference (such as mastoid reference) on local power and ensure that the spatial distribution of the extracted δ-α band power ratio can reflect the true cortical activity.

[0038] In some embodiments, the preprocessing further includes segmenting the original EEG signals of multiple channels to obtain multiple EEG signal segments corresponding to the original EEG signals of multiple channels. In some embodiments, the original EEG signals of multiple channels are divided into multiple EEG signal segments with equal time lengths. For example, the time length of each EEG signal segment may be 1 s to 5 s.

[0039] Step S200: Extract features from the target EEG signals of multiple channels to obtain a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set. Each element in the δ-α band power ratio set represents the degree of cooperation between the activities of the δ band and the α band in the target EEG signals. Each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target EEG signals. Each element in the phase lag index set represents the degree of phase synchronization of the target EEG signals of two channels.

[0040] For the set of δ-α band power ratios, when the values of each element in the set of δ-α band power ratios are relatively large, it indicates that the slow wave (δ band) activity is relatively enhanced and the fast wave (α band) activity is weakened. The increase in δ waves is usually associated with the low arousal or inhibitory state of the brain, and the decrease in α waves may indicate that the brain cannot enter a relaxed state and is in a state of continuous tension or anxiety. The increase in the δ-α power ratio may reflect the abnormal neuroregulation function of the brain, further indicating that the emotional state of the object to be analyzed is more likely to be in a depressive, anxious or other abnormal emotional state. Conversely, when the values of each element in the set of δ-α band power ratios are relatively small, the possibility that the emotional state of the object to be analyzed is in an abnormal emotional state such as depression is relatively small. Moreover, for the power of a single frequency band, such as the δ band or the α band, it is easily affected by individual baseline variations, environmental noise or physiological artifacts, and it is difficult to distinguish normal physiological fluctuations from pathological activities. However, the elements in the set of δ-α band power ratios are normalized by cross-band energy, enhancing the sensitivity to pathological abnormal emotional states such as depression. In other words, the elements in the set of δ-α band power ratios have better recognition of pathological abnormal emotional states such as depression compared to the power of a single frequency band.

[0041] For the set of adaptive similarity tolerance fuzzy entropy, when the values of each element in the set of adaptive similarity tolerance fuzzy entropy are relatively large, it indicates an increase in the complexity and disorder of the target EEG signal, and the possibility that the emotional state of the object to be analyzed is in an abnormal emotional state such as depression is relatively large. Conversely, when the values of each element in the set of adaptive similarity tolerance fuzzy entropy are relatively small, the possibility that the emotional state of the object to be analyzed is in an abnormal emotional state such as depression is relatively small. Moreover, traditional fuzzy entropy is vulnerable to baseline drift or noise interference due to fixed tolerance parameters, which easily leads to bias in complexity assessment. However, each element in the set of adaptive similarity tolerance fuzzy entropy can adaptively adjust the similarity tolerance threshold to more accurately capture the abnormal local activity regularity related to abnormal emotional states such as depression, overcoming the limitation of the sensitivity of traditional fuzzy entropy to parameters.

[0042] For the set of phase lag indices, the value range of each element in the set of phase lag indices is 0 to 1. When the values of each element in the set of phase lag indices are larger, the stronger the phase synchronization of the target EEG signals of the two channels. Enhanced phase synchronization may reflect excessive functional connectivity between certain brain regions, which may be related to rumination and difficulty in emotional regulation during abnormal emotional states such as depression.

[0043] Therefore, in the embodiments of the present application, from the dimensions of the frequency domain, time domain, and spatial domain, three sets are preferably selected for abnormal emotional states to more accurately characterize the characteristics related to abnormal emotional states in the target EEG signal, providing rich feature information for accurately identifying the emotional state of the object to be analyzed subsequently, and avoiding the limitations of single features and traditional features in characterizing emotional states.

[0044] In some embodiments, as Figure 2 shown, feature extraction is performed on the target EEG signals of multiple channels to obtain a set of δ-α band power ratios, including:

[0045] Step S210: Decompose the target EEG signal of each channel into multiple frequency sub-bands, and select the δ band and the α band from the multiple frequency sub-bands;

[0046] Step S220: For the target EEG signal of each channel, determine the first band relative power of the δ band and the second band relative power of the α band;

[0047] Step S230: Based on the first band relative power and the second band relative power, determine the δ-α band power ratio of the target EEG signal of each channel.

[0048] Among them, the multiple δ-α band power ratios corresponding to the target EEG signals of multiple channels form a set of δ-α band power ratios.

[0049] In the embodiments of the present application, the δ-α band power ratio can more accurately characterize the degree of cooperation between the activities of the δ band and the α band in the target EEG signal of each channel. Based on this, in the embodiments of the present application, the target EEG signal is decomposed and the δ band and the α band are selected, and then based on the ratio of the relative powers of the δ band and the α band in the target EEG signal of each channel, the δ-α band power ratio is obtained.

[0050] In some embodiments, for the above step S210, that is, decomposing the target EEG signal of each channel into multiple frequency sub-bands, includes: decomposing the target EEG signal into multiple frequency sub-bands by using wavelet packet decomposition, and the multiple frequency sub-bands include a δ band (1 Hz to 4 Hz), a θ band (4 Hz to 8 Hz), an α band (8 Hz to 13 Hz), and a β band (13 Hz to 30 Hz).

[0051] In some embodiments, as Figure 3 shown, for the above step S220, that is, for the target EEG signal of each channel, determining the first band relative power of the δ band and the second band relative power of the α band, includes:

[0052] Step S221: Sum the absolute powers of the multiple frequency sub-bands of the target EEG signal of each channel to obtain the total power of the multiple frequency sub-bands of the target EEG signal;

[0053] Step S222: Based on the absolute power of the δ band and the total power of the target EEG signal of each channel, determine the first band relative power;

[0054] Step S223: Determine the second band relative power based on the absolute power and total power of the α band of the target EEG signal for each channel.

[0055] In some embodiments, for the above steps S222 and S223, the calculation methods of the first band relative power and the second band relative power are shown in Formula 1 and Formula 2.

[0056]

[0057] Among them, P(δ) represents the first band relative power, P d represents the absolute power of the δ band, P total represents the total power of multiple frequency sub-bands of the target EEG signal, P(α) represents the second band relative power, P α represents the absolute power of the α band.

[0058] In some embodiments, for the above step S230, based on the first band relative power and the second band relative power, determine the δ-α band power ratio of the target EEG signal for each channel, including: dividing the first band relative power by the second band relative power to obtain the δ-α band power ratio. Therefore, the calculation method of the δ-α band power ratio can be shown in Formula 3.

[0059]

[0060] Among them, DAR represents the δ-α band power ratio.

[0061] In some embodiments, the δ-α band power ratio set can be expressed as [DAR1, DAR2,..., DAR k , DAR1 to DAR k respectively represent the δ-α band power ratios of the 1st channel to the kth channel. k represents the total number of channels of the target EEG signal. Among them, when k = 128, the δ-α band power ratio set includes the δ-α band power ratios of 128 channels.

[0062] In some embodiments, as Figure 4 shown, perform feature extraction on the target EEG signals of multiple channels to obtain an adaptive similarity tolerance fuzzy entropy set, including:

[0063] Step S240: Determine the one-dimensional discrete sequence of the target EEG signal for each channel;

[0064] Step S250: Based on the one-dimensional discrete sequence, determine the similarity tolerance value corresponding to the target EEG signal for each channel;

[0065] Step S260: Determine multiple first reconstructed sequences under the first reconstructed space dimension and multiple second reconstructed sequences under the second reconstructed space dimension based on the first reconstructed space dimension, the second reconstructed space dimension, and the one-dimensional discrete sequence.

[0066] Step S270: Determine the first similarity of the target EEG signal of each channel based on the multiple first reconstructed sequences and the similarity tolerance value, and determine the second similarity of the target EEG signal of each channel based on the multiple second reconstructed sequences and the similarity tolerance value.

[0067] Step S280: Determine the adaptive similarity tolerance fuzzy entropy corresponding to the target EEG signal of each channel based on the first similarity and the second similarity.

[0068] Among them, the set of adaptive similarity tolerance fuzzy entropy includes the adaptive similarity tolerance fuzzy entropy corresponding to the target EEG signals of multiple channels.

[0069] In some embodiments of the present application, reconstruction is performed based on the one-dimensional discrete sequence, the first reconstructed space dimension, and the second reconstructed space dimension to obtain multiple first reconstructed sequences and multiple second reconstructed sequences. Then, based on the multiple first reconstructed sequences, the multiple second reconstructed sequences, and the similarity tolerance value, the first similarity corresponding to the first reconstructed space dimension and the second similarity of the second reconstructed space dimension are obtained. Finally, based on the change amplitude between the first similarity and the second similarity, the complexity of the one-dimensional discrete sequence is judged, and further the complexity of each target EEG signal is judged.

[0070] It should be noted that if the change amplitude between the first similarity and the second similarity is small, the one-dimensional discrete sequence has strong regularity, indicating that the complexity of each target EEG signal is low. On the contrary, if the change amplitude between the first similarity and the second similarity is large, the one-dimensional discrete sequence has strong randomness, indicating that the complexity of each target EEG signal is high.

[0071] In some embodiments, for the above step S240, the one-dimensional discrete sequence of the target EEG signal of each channel can be expressed as U, U = {u(i), i = 0, 1, 2, 3,..., N}, where u(i) represents the i-th element in the one-dimensional discrete sequence, and N represents the total number of sampling points in the one-dimensional discrete sequence. For the one-dimensional discrete sequence, each element corresponds to a sampling point, and multiple sampling points are arranged in chronological order. Each element represents the voltage value measured at a specific time point. These voltage values are usually expressed in microvolts (μV), reflecting the intensity of brain activity at that time point.

[0072] In some embodiments, for the above step S250, determining the similarity tolerance value corresponding to the target EEG signal of each channel based on the one-dimensional discrete sequence includes:

[0073] Determine the average value of each element in the one-dimensional discrete sequence as the first average value;

[0074] Determine the squared difference between each element in the one-dimensional discrete sequence and the first average value;

[0075] Determine the average value of multiple squared differences corresponding to multiple elements in the one-dimensional discrete sequence as the second average value;

[0076] Determine the similarity tolerance value based on the square root of the second average value.

[0077] In some embodiments of the present application, the above method determines the similarity tolerance value based on the overall standard deviation of the one-dimensional discrete sequence, so that the similarity tolerance value can change dynamically according to the changes of the elements in the one-dimensional discrete sequence, making the similarity tolerance value have better adaptability and improving the influence of the differences of different signals on the fuzzy entropy. Moreover, when calculating the subsequent adaptive similarity tolerance fuzzy entropy using this similarity tolerance value, the adaptive similarity tolerance fuzzy entropy can be adaptively adjusted according to each target EEG signal.

[0078] In some embodiments, the calculation method of the similarity tolerance value corresponding to the target EEG signal of each channel can be as shown in Formula Four and Formula Five.

[0079]

[0080] Among them, r represents the similarity tolerance value, u(i) represents the i-th element in the one-dimensional discrete sequence, represents the mean value of multiple elements in the one-dimensional discrete sequence, and N represents the total number of sampling points in the one-dimensional discrete sequence.

[0081] In some embodiments, for the above step S260, based on the first reconstruction space dimension, the second reconstruction space dimension, and the one-dimensional discrete sequence, determine multiple first reconstruction sequences under the first reconstruction space dimension and multiple second reconstruction sequences under the second reconstruction space dimension, so as to better analyze the internal structure and dynamic characteristics of the one-dimensional discrete sequence based on the multiple first reconstruction sequences and multiple second reconstruction sequences under different space dimensions.

[0082] In some embodiments, the first reconstructed space dimension may be m. The method for determining a plurality of first reconstructed sequences based on the first reconstructed space dimension and the one-dimensional discrete sequence may be to divide the N elements in the one-dimensional discrete sequence, and m adjacent elements form a first reconstructed sequence. When 1 ≤ i ≤ N - m, the one-dimensional discrete sequence may be divided into N - m first reconstructed sequences. The first reconstructed sequence may be represented as X(i), where X(i) = [u(i), u(i + 1), …, u(i + m - 1)] - u0(i), i = 1, 2, …, N - m + 1. Among them, u0(i) is the reference mean value of the reconstructed vector, and the calculation method of u0(i) may be as shown in Equation 6:

[0083]

[0084] It should be noted that the principle of the method for determining a plurality of second reconstructed sequences based on the second reconstructed space dimension and the one-dimensional discrete sequence is the same as the principle of determining a plurality of reconstructed sequences, and will not be elaborated here.

[0085] In some embodiments, the second reconstructed space dimension may be m + 1. In this way, the second reconstructed space dimension and the first reconstructed space dimension are adjacent space dimensions. Based on the reconstructed sequences under two adjacent space dimensions, an appropriate embedding dimension can be determined to accurately reveal the dynamic characteristics of the one-dimensional discrete sequence in the reconstructed phase space.

[0086] In some embodiments, for step S270 above, determining the first similarity of the target EEG signal of each channel based on a plurality of first reconstructed sequences and a similarity tolerance value, and determining the second similarity of the target EEG signal of each channel based on a plurality of second reconstructed sequences and a similarity tolerance value, includes:

[0087] Determine the maximum value of the distances between each first reconstructed sequence and other first reconstructed sequences as the first maximum value, and determine the first fuzzy membership degree between each first reconstructed sequence and other first reconstructed sequences based on the first maximum value and the similarity tolerance value;

[0088] Determine the first similarity based on the average value of the plurality of first fuzzy membership degrees respectively corresponding to each of the plurality of first reconstructed sequences;

[0089] Determine the maximum value of the distances between each second reconstructed sequence and other second reconstructed sequences as the second maximum value, and determine the second fuzzy membership degree between each second reconstructed sequence and other second reconstructed sequences based on the second maximum value and the similarity tolerance value;

[0090] Determine the second similarity based on the average value of the plurality of second fuzzy membership degrees respectively corresponding to each of the plurality of second reconstructed sequences.

[0091] In some embodiments of the present application, the calculation method of the maximum value of the distance between the first reconstruction sequence X(i) and other first reconstruction sequences X(j) can be as shown in Formula Seven:

[0092]

[0093] wherein, the value range of i is from 1 to (N - m + 1), and j is different from i.

[0094] It should be noted that for each value of i from 1 to (N - m + 1), when determining the maximum value of the distance between each first reconstruction sequence and other first reconstruction sequences, it is necessary to calculate the maximum value of the distance between the first reconstruction sequence and the other (N - m - 1) first reconstruction sequences respectively.

[0095] In some embodiments of the present application, the calculation methods of the first similarity can be as shown in Formulas Eight to Ten:

[0096]

[0097] wherein, represents the first fuzzy membership degree between the first reconstruction sequence X(i) and other first reconstruction sequences X(j) when the reconstruction space dimension is m, represents the average value of corresponding to each i, and φ m (r) represents the first similarity corresponding to multiple first reconstruction sequences.

[0098] It should be noted that the calculation method of the second similarity is the same as that of the first similarity, and will not be elaborated here. In the embodiments of the present application, φ m+1 (r) represents the second similarity.

[0099] In some embodiments, for step S280, the calculation method of the adaptive similarity tolerance fuzzy entropy can be as shown in Formula Eleven:

[0100]

[0101] wherein, ASTFE represents the adaptive similarity tolerance fuzzy entropy.

[0102] In some embodiments, the adaptive similarity tolerance fuzzy entropy set can be expressed as [ASTFE1, ASTFE2,..., ASTFE k , and ASTFE1 to ASTFE k respectively represent the adaptive similarity tolerance fuzzy entropy of the 1st channel to the kth channel. k represents the total number of channels of the target electroencephalogram signal. Among them, when k = 128, the adaptive similarity tolerance fuzzy entropy set includes the adaptive similarity tolerance fuzzy entropy of 128 channels.

[0103] In some embodiments, feature extraction is performed on the target electroencephalogram (EEG) signals of multiple channels to obtain a set of phase lag indices, including:

[0104] Determine the phase lag index between the signals of each pair of channels in the target EEG signals of multiple channels;

[0105] Based on the phase lag indices of multiple channels, construct a phase lag index matrix;

[0106] Perform duplicate removal on the matrix elements of the phase lag index matrix to obtain a set of phase lag indices.

[0107] In some embodiments, the calculation method of the phase lag index PLI between the signals of each pair of channels can be as shown in Equation (12):

[0108]

[0109] where t n represents the nth time point, represents the phase difference between the signals of two channels at time t n ; sign(x) is a sign function that outputs 1 when the independent variable x is positive, outputs -1 when the independent variable x is negative, and outputs 0 when the independent variable x is 0; N represents the number of time points.

[0110] For the target EEG signals of multiple channels, a phase lag index matrix can be constructed to represent the synchronization degree between the signals of different channels in the target EEG signals. The phase lag index matrix P can be expressed as:

[0111]

[0112] where k represents the total number of channels of the target EEG signals.

[0113] In some embodiments, performing duplicate removal on the matrix elements of the phase lag index matrix to obtain a set of phase lag indices includes:

[0114] Determine a diagonal line based on multiple diagonal elements of the phase lag index matrix, where each of the multiple diagonal elements has the same row number as its column number;

[0115] Remove the multiple diagonal elements on the diagonal line and multiple elements on one side of the diagonal line;

[0116] The multiple elements on the other side of the remaining diagonal line form a set of phase lag indices.

[0117] In some embodiments, the set of phase lag indices can be expressed as [PLI1, PLI2,..., PLI q, q = k*(k - 1) / 2. When k is 128, the phase lag index set includes 8,256 - dimensional phase lag index features.

[0118] Step S300, fuse the δ - α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set to obtain a fused feature set.

[0119] In some embodiments, vector concatenation can be used to fuse the three sets, that is, the elements in the three sets are concatenated in order. For example, after fusion using vector concatenation, the fused feature set F can be expressed as [PLI1, PLI1,...., PLI q , ASTFE1, ASTFE2,,..., ASTFE k， DAR1, DAR2,..., DAR k . Therefore, the fused feature set F includes (q + 2k) - dimensional feature elements.

[0120] Step S400, based on the fused feature set, perform emotion state recognition on the object to be analyzed to obtain the emotion state recognition result of the object to be analyzed.

[0121] In the embodiments of the present application, since the fused feature set can comprehensively consider information in multiple aspects such as the frequency band power ratio, complexity, and phase synchronization degree of the target EEG signal, it can improve the overall characterization ability of EEG signal features, especially the characterization ability of non - linear EEG signals. Providing a more comprehensive and rich information basis for emotion state recognition based on these fused features avoids the one - sidedness of a single feature set in emotion state recognition, thus helping to improve the accuracy of emotion state recognition.

[0122] In some embodiments, as Figure 5 shown, for the above - mentioned step S400, that is, perform emotion state recognition on the object to be analyzed to obtain the emotion state recognition result of the object to be analyzed, including:

[0123] Step S410, select multiple target elements from the fused feature set, and the emotion discrimination ability of the multiple target elements is greater than that of other elements in the fused feature set;

[0124] Step S420, input the multiple target elements into a preset classification model to obtain the classification recognition results of the target EEG signals of multiple channels of the object to be analyzed, and obtain the emotion state recognition result based on the classification recognition results.

[0125] In some embodiments of the present application, a plurality of target elements are selected from the fusion feature set, and the emotion discrimination ability of the plurality of target elements is greater than that of other elements in the fusion feature set, so as to further strengthen the representation of the target EEG signals and reduce the amount of data to be recognized, thereby facilitating improving the accuracy and efficiency of the emotion state recognition result obtained by recognition based on the plurality of target elements.

[0126] In some embodiments, as Figure 6 shown, selecting a plurality of target elements from the fusion feature set includes:

[0127] Step S411, determining the emotion discrimination ability coefficient corresponding to each element in the fusion feature set;

[0128] Step S412, sorting the plurality of elements in the fusion feature set according to the absolute value of the corresponding emotion discrimination ability coefficient, and determining the plurality of elements with the front emotion discrimination ability coefficient as target elements from the sorted plurality of elements.

[0129] In some embodiments of the present application, the discrimination ability of each element for the emotion state is quantified as an emotion discrimination ability coefficient, and sorting and selection are performed based on the emotion discrimination ability coefficient, that is, sorting the plurality of elements in the fusion feature set from largest to smallest according to the emotion discrimination ability coefficient, and selecting the plurality of front elements to obtain a plurality of target elements, which is beneficial to improving the efficiency and accuracy of selecting the plurality of target elements.

[0130] In some embodiments, for the above step S411, determining the emotion discrimination ability coefficient corresponding to each element in the fusion feature set includes:

[0131] Obtaining the fusion feature set of each of a plurality of analyzed objects, the plurality of analyzed objects including a first quantity of first-type analyzed objects and a second quantity of second-type analyzed objects;

[0132] Taking the difference between each element of the fusion feature set of each first-type analyzed object and the same element in the fusion feature set of each second-type analyzed object to obtain a first difference, and taking the difference between the label of the first-type analyzed object and the label of the second-type analyzed object to obtain a second difference;

[0133] Based on the first difference corresponding to each element of the fusion feature set of each of the first quantity of first-type analyzed objects and the second difference corresponding to the first difference, determining a third quantity where the first difference and the corresponding second difference are both positive or negative, and a fourth quantity where the first difference and the corresponding second difference are not both positive or negative;

[0134] Based on the first quantity, the second quantity, the third quantity, and the fourth quantity, determine the emotion discrimination ability coefficient corresponding to each element of the fusion feature set.

[0135] In some embodiments, the analyzed object refers to an object whose emotional state has been identified through electroencephalogram signals. In some embodiments, the emotional state of the first type of analyzed object can be an abnormal emotional state such as depression or anxiety, and the emotional state of the second type of analyzed object can be a normal emotional state such as non-depression or non-anxiety.

[0136] In the process of calculating the emotion discrimination ability coefficient of each element in the fusion feature set, it is necessary to find the difference between the i-th element in the fusion feature set of each of the first quantity of the first type of analyzed objects and the i-th element in the fusion feature set of each of the second quantity of the second type of analyzed objects, obtaining a plurality of first differences.

[0137] In some embodiments, the label of the first type of analyzed object can be set to +1, and the label of the second type of analyzed object can be set to -1. After calculating each first difference, the difference between the label of the first type of analyzed object corresponding to each first difference and the label of the second type of analyzed object can be determined as the second difference.

[0138] In some embodiments, the calculation method of the emotion discrimination ability coefficient can be as shown in Equation XIII:

[0139]

[0140] where τ i represents the emotion discrimination ability coefficient, p c represents the third quantity, p d represents the fourth quantity, p m represents the first quantity, p n represents the second quantity.

[0141] In some embodiments, based on obtaining the emotion discrimination ability coefficient corresponding to each element in the fusion feature set, sort the multiple elements in the fusion feature set of the object to be analyzed according to the magnitude of the absolute value of the corresponding emotion discrimination ability coefficient, obtaining a sorted fusion feature set.

[0142] In some embodiments, determining multiple elements with a higher emotion discrimination ability coefficient from the sorted multiple elements includes: inputting the multiple elements of the sorted fusion feature set into a classifier to obtain corresponding classification and recognition results, and determining the target quantity corresponding to the multiple target features according to the convergence of each classification and recognition result.

[0143] Each classification recognition result can correspond to a classification recognition accuracy rate, which is used to measure the classification performance of the classifier based on the elements of this dimension in the classification task. After obtaining the classification recognition accuracy rates corresponding to the elements of each dimension, the corresponding convergence coefficient can be determined, which is used to measure whether the classification performance of the elements converges to a stable value. When the convergence coefficient is less than a certain threshold, it indicates that the classification recognition result converges and the classification performance of the elements is relatively stable.

[0144] In some embodiments, for a fusion feature set with (q + 2k) - dimensional features, the (q + 2k) - dimensional features of the fusion feature set are respectively input into the classifier to obtain the corresponding classification recognition accuracy rates: Acc1, Acc2, Acc3,..., Acc n ,…,Acc q+2k , then the convergence coefficient As of the n - th - dimensional feature n can be expressed as:

[0145]

[0146] where M represents the convergence refinement degree.

[0147] In some embodiments, M can be 8 to 12. Exemplarily, M is 10.

[0148] In some embodiments, for the preset classification model in the above - mentioned step S420, it can be one or more of a k - nearest neighbor (KNN) model, a support vector machine (SVM), a decision tree (DT), and a Naive Bayes (NB).

[0149] In practical applications, by using the method for recognizing an emotional state based on electroencephalogram signals in the embodiments of the present application, the accuracy rate, specificity, and sensitivity for recognizing the emotional state of electroencephalogram signals reach 97.65%, 96.95%, and 98.54% respectively, which greatly improves the accuracy of recognizing the emotional state based on electroencephalogram signals compared with the prior art.

[0150] Please refer to Figure 7 , the embodiments of the present application further provide an apparatus 500 for recognizing an emotional state based on electroencephalogram signals. The apparatus includes an acquisition module 501, a feature extraction module 502, a feature fusion module 503, and a recognition result acquisition module 504.

[0151] The acquisition module 501 is configured to acquire target electroencephalogram signals of multiple channels of an object to be analyzed.

[0152] The feature extraction module 502 is used to extract features from the target electroencephalogram (EEG) signals of multiple channels, obtaining a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set. Each element in the δ-α band power ratio set represents the degree of cooperation between the enhanced activity in the δ band and the suppression in the α band of the target EEG signal. Each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target EEG signal. Each element in the phase lag index set represents the degree of phase synchronization between the target EEG signals of two channels.

[0153] The feature fusion module 503 is used to fuse the δ-α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set to obtain a fused feature set.

[0154] The recognition result acquisition module 504 is used to perform emotion state recognition on the object to be analyzed based on the fused feature set, obtaining the emotion state recognition result of the object to be analyzed.

[0155] The specific implementation manner of the emotion state recognition device 500 is basically the same as the specific embodiments of the above-mentioned emotion state recognition method, and will not be elaborated here.

[0156] Please refer to Figure 8 , this application embodiment also provides an electronic device. The electronic device includes a memory 601, a processor 602, and a computer program stored on the memory 601 and executable on the processor 602. When the processor executes the computer program, it implements the above-mentioned emotion state recognition method based on EEG signals. The electronic device can be any intelligent terminal including a tablet computer, an in-vehicle computer, etc. The memory 601 and the processor 602 can be connected through a communication bus 603.

[0157] In this embodiment, the above-mentioned electronic device can be a related system or program for controlling a brain-computer interface and a brain EEG neural network model to execute an EEG processing flow, such as a server. The communication bus 603 mentioned in the above-mentioned electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 603 can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience of representation, Figure 8 only a thick line is used to represent it in [reference], but it does not mean that there is only one bus or one type of bus.

[0158] The memory 601 may include a random access memory (RAM), or may include a non-volatile memory, such as at least one magnetic disk memory. Optionally, the memory may also be at least one storage device located far from the above-mentioned processor 602.

[0159] The above-mentioned processor 602 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0160] In another embodiment of the present application, a computer-readable storage medium is further provided. A computer program is stored in the computer-readable storage medium. When the computer program runs on a computer, the computer is caused to execute the method for recognizing an emotional state based on an electroencephalogram signal described in the above embodiment.

[0161] The description of the above embodiments is only used to help understand the technical solution and its core idea of the present application; those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements on some of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for recognizing emotional states based on electroencephalogram signals, characterized in that, The method includes: Obtaining target electroencephalogram (EEG) signals of multiple channels of an object to be analyzed; Performing feature extraction on the target EEG signals of multiple channels to obtain a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set. Each element in the δ-α band power ratio set represents the degree of cooperation between the activities of the δ band and the α band in the target EEG signals. Each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target EEG signals. Each element in the phase lag index set represents the degree of phase synchronization of the target EEG signals of two channels; Fusing the δ-α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set to obtain a fused feature set; Based on the fused feature set, performing emotion state recognition on the object to be analyzed to obtain an emotion state recognition result of the object to be analyzed.

2. The method according to claim 1, wherein The performing feature extraction on the target EEG signals of multiple channels to obtain a δ-α band power ratio set includes: Decomposing the target EEG signal of each channel into multiple frequency sub-bands, and selecting the δ band and the α band from the multiple frequency sub-bands; For the target EEG signal of each channel, determining a first band relative power of the δ band and a second band relative power of the α band; Based on the first band relative power and the second band relative power, determining a δ-α band power ratio of the target EEG signal of each channel; wherein, the multiple δ-α band power ratios corresponding to the target EEG signals of multiple channels form the δ-α band power ratio set.

3. The method according to claim 2, wherein The determining the first band relative power of the δ band and the second band relative power of the α band includes: Summing the absolute powers of the multiple frequency sub-bands of the target EEG signal of each channel to obtain the total power of the multiple frequency sub-bands of the target EEG signal; Based on the absolute power of the δ band and the total power of the target EEG signal of each channel, determining the first band relative power; Based on the absolute power of the α band and the total power of the target EEG signal of each channel, determining the second band relative power.

4. The method according to claim 1, characterized in that, The performing feature extraction on the target EEG signals of multiple channels to obtain an adaptive similarity tolerance fuzzy entropy set includes: Determining a one-dimensional discrete sequence of the target EEG signal of each channel; Based on the one-dimensional discrete sequence, determining a similarity tolerance value corresponding to the target EEG signal of each channel; Based on a first reconstruction space dimension, a second reconstruction space dimension, and the one-dimensional discrete sequence, determining multiple first reconstruction sequences in the first reconstruction space dimension and multiple second reconstruction sequences in the second reconstruction space dimension; Based on the multiple first reconstruction sequences and the similarity tolerance value, determining a first similarity of the target EEG signal of each channel, and based on the multiple second reconstruction sequences and the similarity tolerance value, determining a second similarity of the target EEG signal of each channel; Based on the first similarity and the second similarity, determine the adaptive similarity tolerance fuzzy entropy corresponding to the target EEG signal of each channel; wherein, the set of adaptive similarity tolerance fuzzy entropies includes the adaptive similarity tolerance fuzzy entropies corresponding to the target EEG signals of multiple channels.

5. The method according to claim 4, wherein The determining the first similarity based on multiple first reconstruction sequences and the similarity tolerance value, and determining the second similarity based on multiple second reconstruction sequences and the similarity tolerance value includes: Determine the maximum value of the distances between each first reconstruction sequence and other first reconstruction sequences as the first maximum value, and based on the first maximum value and the similarity tolerance value, determine the first fuzzy membership degree between each first reconstruction sequence and other first reconstruction sequences; Based on the average value of the multiple first fuzzy membership degrees respectively corresponding to each of the multiple first reconstruction sequences, determine the first similarity; Determine the maximum value of the distances between each second reconstruction sequence and other second reconstruction sequences as the second maximum value, and based on the second maximum value and the similarity tolerance value, determine the second fuzzy membership degree between each second reconstruction sequence and other second reconstruction sequences; Based on the average value of the multiple second fuzzy membership degrees respectively corresponding to each of the multiple second reconstruction sequences, determine the second similarity.

6. The method according to claim 4, wherein The determining the similarity tolerance value corresponding to the target EEG signal of each channel based on the one-dimensional discrete sequence includes: Determine the average value of the elements in the one-dimensional discrete sequence as the first average value; Determine the square difference between each element in the one-dimensional discrete sequence and the first average value; Determine the average value of the multiple square differences respectively corresponding to the multiple elements in the one-dimensional discrete sequence as the second average value; Based on the square root of the second average value, determine the similarity tolerance value.

7. The method according to claim 1, wherein The performing emotion state recognition on the object to be analyzed based on the fusion feature set to obtain the emotion state recognition result of the object to be analyzed includes: Select multiple target elements from the fusion feature set, and the emotion discrimination ability of the multiple target elements is greater than that of other elements in the fusion feature set; Input the multiple target elements into a preset classification model to obtain the classification recognition result of the target EEG signals of multiple channels of the object to be analyzed, and based on the classification recognition result, obtain the emotion state recognition result.

8. The method according to claim 7, wherein The selecting multiple target elements from the fusion feature set includes: Determine the emotion discrimination ability coefficient corresponding to each element in the fusion feature set; Sort the multiple elements in the fusion feature set according to the absolute value of the corresponding emotion discrimination ability coefficient, and determine the elements with the top emotion discrimination ability coefficients among the sorted multiple elements as the target elements.

9. An emotional state recognition device based on electroencephalogram signals, characterized in that, Includes: An acquisition module, configured to acquire the target EEG signals of multiple channels of the object to be analyzed; A feature extraction module, configured to extract features from the target electroencephalogram signals of multiple channels, and obtain a δ-α band power ratio set, an adaptive similarity tolerance fuzzy entropy set, and a phase lag index set. Each element in the δ-α band power ratio set represents the degree of cooperation between the activities of the δ band and the α band in the target electroencephalogram signal. Each element in the adaptive similarity tolerance fuzzy entropy set represents the complexity of the target electroencephalogram signal. Each element in the phase lag index set represents the degree of phase synchronization of the target electroencephalogram signals of two channels; A feature fusion module, configured to fuse the δ-α band power ratio set, the adaptive similarity tolerance fuzzy entropy set, and the phase lag index set to obtain a fused feature set; An identification result acquisition module, configured to perform emotion state identification on the object to be analyzed based on the fused feature set, and obtain an emotion state identification result of the object to be analyzed.

10. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, the method according to any one of claims 1-8 is implemented.

Images