A method for emotion recognition based on DHOG causal matrix image

Abstract

The present application relates to the field of emotion recognition, and proposes an emotion recognition method based on DHOG causal matrix image, which comprises a causal matrix construction module, a channel selection module, a causal matrix image generation module, a DHOG feature extraction module and an emotion recognition module. First, a causal relationship matrix of emotion is constructed according to the transfer entropy causal relationship between full-channel electroencephalogram signals, then 10 key causal electrode channels are selected from high to low by using channel weighted causal values, a 10*10 causal matrix is obtained, and a causal matrix image is generated, next, the diagonal gradient features of the causal matrix image are extracted by using a DHOG algorithm, finally, a support vector machine or other classifier is used to classify and recognize in the valence and arousal dimension emotion model. The present application extracts image gradient features with classification ability as emotion state features, and can reduce the complexity of operation and the calculation time by reducing the use of electrodes.

Description

TECHNICAL FIELD

[0001] The application relates to the field of emotion recognition, and proposes an emotion recognition method based on a DHOG causal matrix image, which can classify and recognize the emotional state of an individual. BACKGROUND

[0002] Among numerous brain function imaging technologies, electroencephalogram signals can most directly describe the activity state of the brain by recording the discharge of human cerebral cortex cells, have high time resolution, and can reflect the dynamic changes of the human brain in millisecond units, which is conducive to tracking the dynamic change process of the brain by researchers. The electroencephalogram signals under different emotional states can reflect the activation and information interaction of the brain region, so the emotion recognition research based on the electroencephalogram signals has become a research hotspot in the field, and many cross-disciplines such as psychology, brain science, cognitive science and computer science have made various attempts to reveal the brain information processing mechanism under different emotional states and improve the intelligence and friendliness of the brain-computer interface in actual use. For example, the emotional state of a driver is monitored in real time to give a prompt alarm when the emotional state is abnormal, the emotional experience of a user in the use of a product is understood to facilitate the subsequent improvement of the product, and the like. Therefore, the brain-computer interface based on emotion computing has wide application prospects in many fields, and the emotion recognition research based on the electroencephalogram signals is conducive to further obtaining emotional feedback and improving the applicability of the brain-computer interface.

[0003] The latest achievements of brain science show that the brain does not perform emotion processing in a single brain region during emotional activities, but multiple brain regions interact with and influence each other, and the functional division of the brain to some extent reflects that the brain regions have both independence and synergy when the brain performs emotion processing. For example, under emotional stimulation, the activation degree of the frontal lobe and the temporal lobe of the brain is the highest compared with the whole brain, and there are more information flows out of the frontal lobe brain region. Therefore, exploring the emotion generation mechanism through the information interaction between brain regions under different emotional states is an effective research means.

[0004] At present, the emotion recognition research based on the causal brain network of the electroencephalogram signals has achieved certain results, among which the method of constructing a brain network by using the causal information between the electroencephalogram signals to extract the information interaction between brain regions is widely used, but the possibility of using the causal matrix image of the EEG signal for emotion recognition is ignored. Since the color blocks in the causal matrix image contain key gradient information, the emotion state can also be effectively recognized by extracting the gradient histogram features of the image. Initially, gradient features were widely used in the target detection problem in the image, and the spatial features were effectively extracted by extracting the local structural details of the image to perform machine learning, so the gradient information as a causal image gradient feature provides a new method for accurately recognizing the causal features between the EEG channels. SUMMARY

[0005] The application aims to apply the gradient feature of the causal matrix image to the field of emotion recognition, can effectively classify different emotional states, and proposes an emotion recognition method based on DHOG causal matrix image, and the technical problems to be solved include:

[0006] (1) How to use the relationship between multiple scalp EEG channel pairs to construct a causal connection mode;

[0007] (2) How to select the most relevant EEG channel to emotion;

[0008] (3) How to extract gradient features from the causal matrix image of EEG signals and perform classification and recognition.

[0009] The emotion recognition method based on DHOG causal matrix image provided by the application has the characteristics that it comprises the following technical measures:

[0010] Step one, construct an EEG full-channel causal relationship matrix;

[0011] Step two, channel selection, sort the channel causal values from large to small to get the priority order of the channel, select the top n channels, this process effectively reduces the redundant causal information;

[0012] Step three, construct a causal matrix image, use the normalized transfer entropy value between the key channels selected by the channel selection module to construct a matrix image;

[0013] Step four, extract the diagonal line direction gradient feature in the causal matrix image based on the DHOG algorithm;

[0014] Step five, emotion recognition, send the DHOG histogram feature of the causal image into the SVM classifier for training, and perform emotion recognition on the valence and arousal two-dimensional emotion model.

[0015] Compared with the prior art, the emotion recognition method based on DHOG causal matrix image has the beneficial effects that:

[0016] (1) The transfer entropy causal algorithm is used to measure the causal relationship between EEG signals, and the causal matrix constructed contains the space-time information of EEG data;

[0017] (2) The causal matrix image constructed in this method restores the spatial and functional relationship of the data itself, and the color difference can represent the strength of the causal relationship between signals;

[0018] (3) Channel selection problem, after getting the full channel causal value of each trial of each subject, the transfer entropy value is normalized, then the causal values of the same channel of all subjects are added, and the priority order of the channel is obtained by sorting from large to small, and the first n channels are selected, which effectively reduces the redundant causal information and eliminates some electrodes irrelevant to emotion, so that the processing time and hardware cost are reduced while reducing the number of channels. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 An implementation flowchart of an emotion recognition method based on a DHOG causal matrix image

[0020] Figure 2 A channel selection flowchart

[0021] Figure 3 Causal matrix graphs in four emotional states of HVHA, HVLA, LVHA and LVLA DETAILED DESCRIPTION

[0022] In order to make the purpose, technical scheme and advantages of the present application clearer, the specific embodiments of the present application are further described in detail below with reference to the drawings.

[0023] (1) Emotion recognition method flow, including the following steps:

[0024] Data preprocessing: first, the EEG data needs to be preprocessed, the purpose is to remove the power frequency interference and tail effect of the signal. The EEG signal recording time in DEAP dataset is 63s, the first 3s is baseline data, and the remaining 60s data is used for experiment;

[0025] Channel selection: according to the transfer entropy causal value of the channel from large to small, the priority order of the channel is obtained, and the first 10 channels are selected;

[0026] Constructing causal matrix: calculating the causality between each channel EEG signal by using transfer entropy algorithm, and constructing the causal connection matrix of TE;

[0027] Extracting DHOG features: extracting the traditional gradient and diagonal gradient information of the causal matrix image as emotional features;

[0028] Training network structure model: the final image DHOG feature vector is divided into test set and training set and sent into the classifier, and the emotion classification accuracy rate of the test sample is obtained.

[0029] (2) Channel selection flowchart

[0030] The full channel causal matrix is constructed for each trial of each subject, the transfer entropy value is normalized, then the causal values of the same channel of all subjects are added, the channel causal values are sorted from large to small to obtain the priority order of the channel, and the first 10 channels with high sorting values are selected, which effectively reduces the redundant causal information.

[0031] (3) Analysis of the results of the DEAP dataset data published internationally

[0032] Figure 3 For the TE causal matrix of the first subject in the DEAP dataset under the HVHA, HVLA, LVHA and LVLA four emotional states, the TE value continuously increases from blue to red, it can be observed that the TE value under the LVHA and LVLA emotional states is relatively higher than that under the HVHA and HVLA states, that is, the causal information flow of different brain regions is more under the negative emotional state, that is, the interaction range of the brain regions is wider, forming a larger range of intracerebral information transmission. And the information flow of the brain regions under the positive emotional state is less and more concentrated. That is, the human brain pays more attention to detailed information under the negative state, thereby activating more related brain regions for information processing.

[0033] Although the specific embodiments of the present application are described above in combination with the drawings, it is not a limitation on the protection scope of the present application, and those skilled in the art should understand that various modifications or deformations made by those skilled in the art on the basis of the technical solutions of the present application without creative labor are still within the protection scope of the present application.

Claims

1. A method for emotion recognition based on DHOG causal matrix image, characterized in that, The emotion recognition method comprises a causal matrix construction module, a channel selection module, a causal matrix graph generation module, a DHOG feature extraction module and an emotion recognition module, and the specific implementation process comprises the following steps. Step 1: The causal matrix construction module is used to measure the causal relationship between full-channel EEG signals by using a transfer entropy algorithm, and a full-channel causal relationship matrix of EEG is constructed. Step 2: The channel selection module is used to, based on step 1, construct a full-channel causal matrix for each test of each subject, normalize the transfer entropy value, add the causal values of the same channel of all subjects, sort the channel causal values from large to small, obtain the priority order of the channel, and select the top n channels in the order value, so that the process effectively reduces the redundant causal information. Step 3: The causal matrix graph generation module is used to generate a key causal matrix graph by using the normalized transfer entropy value between the key channels selected by the channel selection module in step 2, that is, the relative information amount transmitted between the n channels. Step 4: The DHOG feature extraction module is used to extract the diagonal histogram of oriented gradients (DHOG) feature in the key causal matrix graph. First, the color and Gamma space of the causal matrix image are normalized, the GRB component of the image is converted into a Gray component, and a gray image is generated. Then, the gradient feature of the matrix network image is extracted by using the HOG algorithm, and the gradient histogram of the pixel points in the horizontal direction and the vertical direction is obtained. Further, the diagonal line gradient feature of the image is extracted, and the gradient histogram of the pixel points in the diagonal line of the image is calculated. Finally, the gradient histograms in the horizontal direction, the vertical direction and the diagonal line are concatenated and combined, and the final DHOG feature vector of the pixel point is obtained. Step 5: The emotion recognition module is used to send the DHOG feature vector of the image into a SVM classifier for training, and perform emotion recognition on the valence and arousal dimension emotion models.

2. The emotion recognition method based on the DHOG causal matrix image according to claim 1, wherein, Since the current research on the causal relationship of the electroencephalogram signal is mostly based on the construction of a causal matrix from full-channel signals, this construction method increases the invalid or weakly related causal relationship to some extent, resulting in many redundant information unrelated to emotions in the causal network. The channel selection operation in step 2 selects the EEG data of the 10 channels most closely related to emotions, which can not only reduce the complexity of the calculation and remove redundant information, but also reduce the operation time and improve the practicability of the system. The transfer entropy causal values are sorted from large to small to obtain the priority order of the channels: FP1, FP2, F3, F4, C4, P3, CP1, CP2, O1 and Oz.

3. The emotion recognition method based on DHOG causal matrix images as described in claim 1, characterized in that, The DHOG features of the causal matrix image are extracted in steps 3 and 4 for emotion recognition operation. Firstly, the gradient of each pixel in horizontal and vertical direction is measured to obtain the gradient modulus and the direction angle of each pixel in the image. Then, the diagonal modulus and the direction angle are extracted in addition to the vertical and horizontal gradient values. In order to encode the local region of the image to ensure the edge information of the image, the cell unit of the image is divided and the gradient direction histogram is counted. The image is divided into blocks, each block is composed of 2x2 cells, each cell is called a cell unit of the image, which contains 8x8 pixels. The image is scanned by blocks, the scanning unit is 8 pixels. The pixel gradient of the image is mapped to the range of 0°-360°, and then averaged to 9 bin intervals. A 9-dimensional feature vector is obtained from each cell, and the feature vector is the gradient histogram of the cell. The DHOG features of all blocks in the image are concatenated to obtain the DHOG feature of the image, which is the final DHOG feature vector of the image used for classification. Finally, the DHOG feature is sent to the SVM classifier for training, and the emotional recognition is performed in the valence and arousal dimensions. ​

Images