Emotion recognition system based on multi-modal physiological and social information
By constructing a multimodal physiological and social information emotion recognition system, combining EEG and peripheral nerve signals, and social information, the problem of insufficient emotional recognition performance in the existing technology is solved, and more accurate emotion recognition is achieved.
Patent Information
- Application Number
- CN202411243948.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-24
- Estimated Expiration
- 2044-09-05
AI Technical Summary
Existing emotion recognition technologies are difficult to effectively utilize EEG signals and peripheral neural network information in different brain regions, and lack the ability to link social environment information with emotions, resulting in insufficient emotional recognition performance.
Using a multimodal physiological and social information emotion recognition system, through brain network module, neural system network module and social network module, combined with physiological signals and social information such as EEG, ECG, EMG, etc., to build a neural system network model from local to global , and use hypergraph neural network for emotion recognition.
The modeling from the local central nervous system to the global nervous system is realized, and combined with peripheral nervous system information is improved, and the social environment information is included through social network models, improving the ability of emotional recognition.
Smart Images

Figure CN119179945A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of emotion recognition, and in particular relates to an emotion recognition system based on multimodal physiological and social information. Background Art
[0002] Emotions are part of human daily life, affecting human cognition, behavior, and decision-making, and are indispensable in social interactions. Emotions affect almost every aspect of human communication, including tone of voice, facial expressions, word choice, breathing, muscle tension, and so on. It has the potential to change the message in essence, as the way of speaking is more important than what is said. Researchers from different fields, including psychology, biology, sociology, and neuroscience, have defined, explained, analyzed, and classified human emotions. In recent years, the availability of various electronic products in our lives has made people spend more and more time on social media, online shopping, online video games, etc. However, most current human-computer interaction (HCI) systems have defects in interpreting and understanding emotional information and lack emotional intelligence. They are unable to recognize human emotional states and cannot use this information for decision-making and action. Solving the problem of lack of rapport between humans and machines is essential for advanced intelligent HCI. With the continuous development of the HCI field, it is urgent to give machines the ability to understand human emotions. To this end, emotion recognition technology provides a feasible way to obtain human emotional states.
[0003] Emotion recognition research mainly includes two aspects: sentiment analysis and emotion recognition. The former performs coarse-grained emotion recognition (usually binary positive and negative or 3-category positive, negative and neutral emotion classification tasks), while the latter involves fine-grained analysis (usually multi-class classification of big data into a larger set of emotion labels, such as more than 4 categories). The number of emotion categories has always been controversial in psychology. Historically, psychologists have two different approaches to model emotions: one is the basic emotion theory, which labels emotions as discrete categories; the other is the multidimensional theory, which classifies emotions into multiple dimensions or scales. The basic emotion theory believes that humans have several basic emotions, such as happiness, sadness, fear, anger, disgust and surprise. Other non-basic emotional states (such as fatigue, anxiety, satisfaction, confusion and frustration) are composed of them. Each type of emotion has a unique internal experience, external manifestation and physiological pattern.
[0004] Existing emotion recognition methods can be divided into two categories: subjective methods (such as emotion recognition based on body movements, facial expressions or voice) and objective methods (such as emotion recognition based on different types of physiological signals). Compared with subjective methods, objective methods based on physiological signals, such as (electroencephalogram, EEG), electrocardiogram (electrocardiogram, ECG), electromyogram (electromyogram, EMG), etc., can eliminate the differences between different subjects and exclude the influence of the subjective consciousness of the subject. Therefore, it has been widely used in processing emotion recognition in recent years.
[0005] In the past few years, deep learning has achieved great success in many fields, such as signal processing, artificial intelligence, and emotion detection, relying on the most modern systems. Convolutional neural networks (CNN), recurrent neural networks (RNN), and graph neural networks (GNN) are commonly used methods in deep learning. Applying deep learning to emotion recognition is one of the main research directions of emotion recognition.
[0006] Li et al. proposed a new EEG emotion recognition method inspired by neuroscience involving the brain's response to different emotions. The method adopts the R2G-STNN method, which consists of a spatiotemporal neural network model with a regional to global hierarchical feature learning process to learn discriminative spatiotemporal EEG features.
[0007] Although a variety of EEG emotion recognition methods have been proposed in recent years, there are still some major issues that need to be studied in depth in order to further improve the performance of emotion recognition. The first problem is how to use the information of EEG signals in different brain regions to improve emotion recognition. Neuroscience research in recent years has shown that human emotions are closely related to a variety of cerebral cortical regions, such as the orbitofrontal cortex, ventromedial prefrontal cortex, and amygdala. Therefore, EEG signals in different brain regions contribute differently to emotion recognition. The second problem is how to use peripheral neural networks to supplement information to brain neural networks. Peripheral neural networks, including ECG, EMG, etc., also contain rich emotional information. Mining and using this information to obtain a higher-dimensional neural network will help improve emotion recognition. The third problem is how to use the social environment information in which people live to improve the ability of emotion recognition. As we all know, the social environment in which people live will also affect their emotions, and people in the same environment often have the same emotions. How to link these social information with emotions is also one of the problems that the present invention hopes to solve. Summary of the invention
[0008] In view of this, the purpose of the present invention is to provide an emotion recognition system based on multimodal physiological and social information. The system can use multimodal physiological signals to model brain neural networks and peripheral neural networks, explore the representation of emotions by brain neural networks and peripheral neural networks, and at the same time use social information to model social network models, and explore the interaction between emotions and social environment.
[0009] An emotion recognition system based on multimodal physiological and social information, including a brain network building module, a neural network building module, a social network building module and an emotion recognition module:
[0010] The brain network building module is used to:
[0011] The EEG is divided according to the location of collection, and the EEG signals of different brain regions are input into the bidirectional long short-term memory network BiLSTM of different scales to learn the spatiotemporal characteristics of different brain regions and build a local brain network model. The spatial features learned from different brain regions are input into the GNN network model to learn the spatial features between brain regions and obtain the feature tensor X corresponding to each brain region. fs The brain network building module introduces a regional attention layer to learn a set of X based on the different roles played by different brain regions in EEG emotion recognition. fs The weight matrix W fs , used to represent the contribution of different brain regions; then the feature tensor X of each brain region whose spatial and temporal features have been extracted is fs Concatenate into feature tensor X r , and X r With W fs Multiply to get X after introducing weights r 'Input into the global GNN_global to get the feature tensor X gCNS , used to characterize the temporal and spatial characteristics within and between different brain regions, reflect the information contained in the central nervous system, and realize the construction of brain network models from local to global;
[0012] The neural network construction module is used to input the preprocessed physiological signals of the peripheral neural network into different BiLSTMs to extract temporal features, and then input them into different GNNs to extract spatial features, and then compare the obtained spatiotemporal features with the feature tensor X obtained by the brain network construction module. gCNS Splicing is performed to obtain the feature X containing the time and space information of the central nervous system and the peripheral nervous system p ;
[0013] The social network building module is used to: pAs node features, we use the social information of different people as the basis for the nodes’ connection edges and input them into the hypergraph neural network, thereby obtaining the HGNN association matrix H. p Perform hypergraph convolution operation to obtain feature vector X e ;
[0014] The emotion recognition module is used to: transform the final feature tensor X e The input is sent to the emotion recognition module, which uses a fully connected layer and an activation function softmax to obtain the final emotion recognition result.
[0015] Preferably, the brain network construction module divides the EEG signal into 9 brain regions.
[0016] Preferably, the social information includes gender, age, and weight information.
[0017] Preferably, the feature vector X is obtained e The methods include:
[0018] The social information is expressed as a one-dimensional vector X sd Represents; the one-dimensional vector X sd As the attribute of the point in HGNN, the feature tensor X p As the feature of the midpoint in HGNN, the hyperedge is generated using the attribute, and the X sd Points with the same corresponding bit values are connected by hyperedges, thus obtaining the association matrix H of HGNN and performing p Perform hypergraph convolution operation, and finally obtain the feature tensor X e .
[0019] Preferably, for the feature tensor X p Perform multiple hypergraph convolution operations in succession, and finally obtain the feature tensor X e .
[0020] Preferably, the obtained feature tensor can also repeat the above operation. In this embodiment, three hypergraph convolution operations are performed. The specific formula of the hypergraph convolution operation is:
[0021] Among them, D v and D e Represents the diagonal matrices of vertex degree and edge degree, W is the weight matrix, Θ p is a learnable parameter and σ is a nonlinear activation function.
[0022] Preferably, the emotion recognition module is also used to train the network, and during the training process, cross entropy is used to calculate the loss function, and Adam optimization is used as the optimization algorithm.
[0023] The present invention has the following beneficial effects:
[0024] 1. The present invention uses EEG to achieve modeling of the central nervous system network from local to global, uses EEG inside different brain regions to obtain local brain neural networks, and uses spatial information between different brain regions to obtain global brain neural networks, thereby achieving modeling from the local central nervous system to the global nervous system network.
[0025] 2. The present invention uses other modalities of physiological signals to supplement the information of the central nervous system network on the basis of constructing the central nervous system network, and uses ECG, EMG and other signals to model the peripheral nervous system network, thereby realizing the modeling from the central neural network to the global nervous system network of the human body.
[0026] 3. After obtaining the human nervous system network, the present invention also uses the individual's social information to realize the construction of a social network, thereby realizing the construction of a network model from a local nervous system network to a global nervous system network, and from an individual to a society. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] Figure 1 A flowchart of an emotion recognition system based on multimodal physiological and social information provided by an embodiment of the present invention.
[0028] Figure 2 A data preprocessing flow chart for different physiological signals provided by an embodiment of the present invention.
[0029] Figure 3 It is a schematic diagram of a model construction module for constructing a central neural network model provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0030] The present invention is described in detail below with reference to the accompanying drawings and embodiments.
[0031] EEG is a technology that records the electrophysiological activity of brain nerve cells in the cerebral cortex or on the surface of the scalp. It captures the electrical signals of neurons by placing electrodes on the scalp. These signals reflect the activity state of the brain, so these electrical signals can be recorded and analyzed by EEG to identify emotions.
[0032] Similar to EEG, ECG is also a technology for recording physiological signals. It captures the heart's electrical signals by placing electrodes on the surface of the skin. These signals reflect the heart's electrophysiological state and function.
[0033] Similarly, EMG is a technology that records the electrical activity of muscles by capturing electrical signals generated by muscle cells through electrodes placed on the surface of the skin or inserted directly into the muscles. These signals reflect the activity status of the muscles.
[0034] This system mainly uses the deep learning network architecture to build each part of the network model. The specific network architectures used include: BiLSTM, GNN, HGNN, and the pytorch framework is used to build the above networks. The specific usage is described as follows:
[0035] BiLSTM is a special recurrent neural network that can process sequence data and maintain long-term memory. BiLSTM takes into account both past and future information, enabling the model to better capture the contextual relationships in sequence data. For physiological signals, the use of BiLSTM can well capture the temporal characteristics of physiological signals.
[0036] GNN is a neural network specially designed for processing graph structure data. Compared with traditional neural networks, GNN can operate directly on the graph and learn the representation of the nodes and edges of the graph. This system uses each channel / individual as a node, which can effectively learn the connection between channels / individuals. For physiological signals, GNN can well capture the spatial features between different channels.
[0037] HGNN is a neural network specifically designed to process hypergraph structured data. Hypergraph is a more complex data structure than traditional graphs, which allows multiple nodes (vertices) to be connected to the same hyperedge at the same time. This structure is very natural when representing many-to-many relationships. Compared with GNN, it can more effectively capture higher-dimensional feature relationships and has considerable advantages in processing multimodal data. This system uses HGNN to construct the social network when generating the social network at the end, which can effectively utilize personal social information and physiological signals to learn higher-dimensional features for emotion recognition.
[0038] An emotion recognition system based on multimodal physiological and social information of the present invention is divided into four parts; the first part mainly considers the comprehensive utilization of EEG information of different brain regions to realize the construction of brain network from local to global. EEG is divided according to the location of collection (the present embodiment divides the EEG of 32 channels into 9 different regions), and the EEG signals of different brain regions are respectively input into the bidirectional long short-term memory (Bidirectional long short-termmemory, BiLSTM) network of different scales to learn the spatiotemporal characteristics of different brain regions and build a local brain network model, and the spatial features learned from different brain regions are input into the GNN network model to learn the spatial features between brain regions, so as to realize the construction of brain network model from local to global. At the same time, considering that different brain regions play different roles in EEG emotion recognition, this part also introduces a regional attention layer to learn a set of weights representing the contribution of brain regions. This part comprehensively utilizes the spatiotemporal information inside the brain region and the spatial information between the brain regions, effectively utilizes the EEG signals between different brain regions, and realizes the construction of brain network from local to global.
[0039] The second part mainly considers using the physiological signals of the peripheral neural network to supplement the information of the emotion recognition system. Similar to the first part, the physiological signals of the peripheral neural network, such as ECG and EMG, are also preprocessed. After preprocessing, the processed ECG and EMG signals are respectively input into different BiLSTM to extract time features, and then the obtained time features are input into GNN to obtain the peripheral neural network model. The obtained peripheral neural network is integrated with the brain neural network of the first part to achieve modeling from the local brain neural network to the global human neural network. This part mainly uses the signals of the peripheral neural network to model the peripheral neural network, and uses the information of the peripheral neural network to supplement the brain neural network, and finally obtains a complete personal neural network model.
[0040] The third part mainly considers using people's social information to model social networks among people. This part takes individuals as nodes in the social network, uses the social information of different people as the basis for the edges connecting nodes, and uses the features extracted from the personal neural network in the second part as node information, which is input into the Hypergraph Neural Network (HGNN) to achieve social network modeling from individuals to the entire group. By learning the topological structure and semantic information of the social network, a feature representation with individual attributes and social attributes is obtained, and this feature representation is used to achieve good emotion recognition.
[0041] The system introduced in this invention builds a comprehensive emotion recognition system from different perspectives. The neural network model based on EEG focuses on capturing the temporal and spatial characteristics of different brain regions and between different brain regions of EEG signals; the neural network model based on multimodality improves the accuracy of recognition by fusing information from different modalities; and the social network construction based on social information further takes social factors into consideration to achieve more comprehensive emotion recognition.
[0042] In this embodiment, the system first acquires information of multiple modalities, including EEG, ECG, EMG, and social information, and then preprocesses the EEG, ECG, and EMG data and extracts features; then, through the network construction module, the features extracted from the data of different modalities are integrated to establish a social network model; finally, emotion recognition is performed on the emotion recognition module.
[0043] The present invention is described in detail below with reference to the accompanying drawings and embodiments:
[0044] Figure 1 This is a flow chart of an emotion recognition system based on multimodal physiological and social information provided by an embodiment of the present invention. The system includes three parts, each of which includes several feature extraction modules and a network construction module, and finally the final emotion recognition is performed through the emotion recognition module.
[0045] In the first part, the system first acquires EEG signals. This embodiment takes the DEAP dataset as an example. The DEAP dataset collects EEG and EMG signals from 32 healthy participants while they watch 40 one-minute music videos. EEG signals are collected in 32 channels according to the international standard 10-20 system, with a collection frequency of 512Hz and downsampled to 128Hz. Therefore, for each subject, the EEG signal is a tensor of size X1 = (40, 32, 60*128). The obtained data is then input into the feature extraction module for feature extraction. The feature extraction module is as follows: Figure 2 shown.
[0046] Figure 2This is a general feature extraction flow chart for different physiological signals provided by an embodiment of the present invention. Taking EEG signals as an example, first, a high-pass filter with a cutoff frequency of 0.5Hz and a low-pass filter with a cutoff frequency of 45Hz are used to remove high and low frequency noise in the EEG signal; then a reference electrode is selected to re-reference the data to remove the baseline drift of the signal; the independent component analysis method is used to screen and eliminate parts that do not belong to the EEG signal, such as eye movement, electrocardiogram, etc.; a suitable window size (such as 1s) is selected to divide the data segments for time analysis; finally, the segmented EEG data is band-pass filtered, and the corresponding cutoff frequency of the band-pass filter is set to obtain the differential entropy features of five different frequency bands: delta (1-4Hz), theta (4-8Hz), alpha (8-13Hz), beta (13-30Hz) and gamma (31-45Hz). For the DEAP data set, the data size of each subject is processed from (40, 32, 60*128) to X fCNS =(40, 32, 60, 5) tensor.
[0047] Input the feature extracted tensor into the model building module, Figure 3 Schematic diagram of a model building module for building a central neural network model provided by an embodiment of the present invention. The module first divides the feature tensor and divides the 32 channels into 9 brain regions. The specific division method is shown in Table 1, which is a channel table included in different brain region divisions.
[0048] Table 1. Channels included in different brain regions
[0049]
[0050]
[0051] The different channel signals of the same brain region are input into the corresponding BiLSTM to extract the time features inside the 9 brain regions. Different BiLSTMs are labeled according to the brain region they are in. Next, taking the frontal lobe as an example, how to extract the time and space features inside the brain region is described. The BiLSTM of the frontal lobe is labeled as BiLSTM_PF. In this embodiment of the present invention, the hidden layer of BiLSTM_PF is set to 60, so the tensor size after BiLSTM_PF is X ft =(40, 4, 120), and then the 9 tensors after extracting the time features are input into the new GNN to extract the spatial features of the brain area. Taking the frontal lobe as an example, the hidden layer of GNN_PF is set to 64, and the tensor size obtained is X fs =(40, 64), considering that different brain regions play different roles in EEG emotion recognition, this embodiment introduces an attention mechanism at the end to learn a set of X fsThe weight matrix W fs , used to represent the contribution of different brain regions, the 9 tensors X that have been extracted with spatial and temporal features fs Splice to size X r =(40, 9, 64) tensor, and X r With W fs Multiply to get X after introducing weights r ' is input into the global GNN_global, and the resulting central neural network model is the GNN_global with 9 vertices. The GNN_global hidden layer is set to 128, so the final size is X gCNS = A tensor of size (40, 128), which contains the temporal and spatial characteristics within and between the nine brain regions, and can well reflect the information contained in the central nervous system.
[0052] The data acquisition in the second part is the same as that in the first part, and peripheral neural network signals such as ECG and EMG are obtained. Taking the DEAP dataset as an example, the DEAP dataset contains the EMG of two channels of the above 32 participants, which are also downsampled to 128Hz. This embodiment adopts a feature extraction module process similar to that in the first part, but does not use bandpass filtering in the last step, and directly obtains the differential entropy feature of EMG, and obtains a size of X fPNS The tensor (40, 2, 60) is input into the model building module in the second part. Similar to the module in the first part, this module first uses BiLSTM_PNS to extract the temporal features of the peripheral neural network, and then uses GNN_PNS to build the peripheral neural network, which can effectively capture the spatial features of the peripheral nervous system. The hidden layer of BiLSTM_PNS is also set to 60, and the hidden layer of GNN_PNS is set to 128, so X can also be obtained. gPNS =(40, 128) size tensor, and finally combine this tensor with the tensor X obtained in the first part gCNS Splice to get a size of X p =(40, 2, 128), which contains both the temporal and spatial information of the central nervous system and the peripheral nervous system.
[0053] The third part first obtains the social information of the participants, including gender, age, weight and other information. For example, the DEAP data set collects information on 32 participants, including 16 boys and 16 girls, with an age range of 19 to 37 years old and an average age of 26.9 years old. All participants are in good physical and mental condition, have no mental illness or brain damage, and are right-handed. Non-data information is digitized and arranged into one-dimensional vectors in a certain order, such as males are marked as 0, females are marked as 1; students are marked as 0, and social personnel are marked as 1. These one-dimensional vectors can represent the social information of the participants. For example, a 19-year-old male student weighing 70kg can be represented by a one-dimensional vector X sd :(0,0,19,70). These one-dimensional vectors are used as the attributes of the HGNN midpoint, and the tensor obtained in the second part is used as the feature of the HGNN midpoint. This embodiment uses attributes to generate hyperedges. Taking the above participants as an example, the X of the 32 participants is sd Points with the same corresponding bit values are connected by hyperedges, thus obtaining the association matrix H of HGNN and performing p Perform hypergraph convolution operation, the specific formula is as follows: Where D v and D e The diagonal matrices representing vertex degree and edge degree, W is the weight matrix, which is a diagonal matrix of 1 by default, Θ p is a learnable parameter, and σ is a nonlinear activation function. The obtained feature tensor can also repeat the above operation. In this embodiment, after three hypergraph convolution operations, the final feature tensor size is a one-dimensional vector X of 64. e .
[0054] In the emotion recognition module in the fourth part, the final feature tensor X e The input is sent to the emotion recognition module, which passes the feature tensor through the fully connected layer and the activation function softmax to obtain the final emotion recognition result. The result classifies the emotion with "0" representing low arousal and "1" representing high arousal.
[0055] After obtaining a complete social network, the network needs to be trained to obtain better results. In this embodiment, the DEAP data set is divided into a training set, a validation set, and a test set according to the subject-independent method. The data in the training set and the validation set contain labels of "0" or "1" as the basis for network optimization. As mentioned above, this embodiment uses "0" to represent low arousal and "1" to represent high arousal. At the same time, a five-fold cross-validation is performed to ensure the accuracy of the results. During the training process, the cross entropy loss function is calculated, and Adam optimization is used as the optimization algorithm.
[0056] In summary, the above are only preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.
Claims
1. An emotion recognition system based on multimodal physiological and social information, characterized in that: Including brain network building module, neural network building module, social network building module and emotion recognition module: The brain network building module is used to: The EEG is divided according to the location of collection, and the EEG signals of different brain regions are input into the bidirectional long short-term memory network BiLSTM of different scales to learn the spatiotemporal characteristics of different brain regions and build a local brain network model. The spatial features learned from different brain regions are input into the GNN network model to learn the spatial features between brain regions and obtain the feature tensor X corresponding to each brain region. fs The brain network building module introduces a regional attention layer to learn a set of X based on the different roles played by different brain regions in EEG emotion recognition. fs The weight matrix W fs , used to represent the contribution of different brain regions; then the feature tensor X of each brain region whose spatial and temporal features have been extracted is fs Concatenate into feature tensor X r , and X r With W fs Multiply to get X after introducing weights r 'Input into the global GNN_global to get the feature tensor X gCNS , used to characterize the temporal and spatial characteristics within and between different brain regions, reflect the information contained in the central nervous system, and realize the construction of brain network models from local to global; The neural network construction module is used to input the preprocessed physiological signals of the peripheral neural network into different BiLSTMs to extract temporal features, and then input them into different GNNs to extract spatial features, and then compare the obtained spatiotemporal features with the feature tensor X obtained by the brain network construction module. gCNS Splicing is performed to obtain the feature X containing the time and space information of the central nervous system and the peripheral nervous system p ; The social network building module is used to: p As node features, we use the social information of different people as the basis for the nodes’ connection edges and input them into the hypergraph neural network, thereby obtaining the HGNN association matrix H. p Perform hypergraph convolution operation to obtain feature vector X e ; The emotion recognition module is used to: transform the final feature tensor X e The input is sent to the emotion recognition module, which uses a fully connected layer and an activation function softmax to obtain the final emotion recognition result.
2. The emotion recognition system based on multimodal physiological and social information as claimed in claim 1, characterized in that: The brain network building module divides the EEG signals into 9 brain regions.
3. The emotion recognition system based on multimodal physiological and social information as claimed in claim 1, characterized in that: The social information includes gender, age, and weight information.
4. The emotion recognition system based on multimodal physiological and social information as claimed in claim 1, characterized in that: Get the feature vector X e The methods include: The social information is represented by a one-dimensional vector X sd Represents; the one-dimensional vector X sd As the attribute of the point in HGNN, the feature tensor X p As the feature of the midpoint in HGNN, the hyperedge is generated using the attribute, and the X sd Points with the same corresponding bit values are connected by hyperedges, thus obtaining the association matrix H of HGNN and performing p Perform hypergraph convolution operation, and finally obtain the feature tensor X e .
5. The emotion recognition system based on multimodal physiological and social information as claimed in claim 4, characterized in that: For the feature tensor X p Perform multiple hypergraph convolution operations in succession, and finally obtain the feature tensor X e .
6. An emotion recognition system based on multimodal physiological and social information as claimed in claim 4 or 5, characterized in that: The obtained feature tensor X p After three hypergraph convolution operations, the specific formula of the hypergraph convolution operation is: Among them, D v and D e Represents the diagonal matrices of vertex degree and edge degree, W is the weight matrix, Θ p is a learnable parameter and σ is a nonlinear activation function.
7. The emotion recognition system based on multimodal physiological and social information as claimed in claim 1, characterized in that: The emotion recognition module is also used to train the network. During the training process, cross entropy is used to calculate the loss function, and Adam optimization is used as the optimization algorithm.
Citation Information
Patent Citations
Gender difference detection method based on electroencephalogram emotion recognition
CN113951883A
Sleep staging method and system based on multi-modal physiological signal fusion
CN115349821A
Emotion recognition method, device and equipment based on hybrid graph network and storage medium
CN115374817A
Emotion recognition method based on multi-modal physiological signal feature fusion
CN117297603A
Method for realizing a multi-channel convolutional recurrent neural network EEG emotion recognition model using transfer learning
US20230039900A1
Cited By
Emotion recognition method, system and terminal based on multi-scale brain network decoding model
CN120661158A
