Emotion recognition method and device based on EEG signals, electronic equipment and medium

Abstract

The embodiment of the present application provides an emotion recognition method and device, electronic device and medium based on EEG signals, which belongs to the field of digital medical technology. The method includes: obtaining target EEG data of a target object, the target EEG data includes multiple target EEG signals; performing frequency domain feature extraction on the target EEG signal to obtain differential entropy features; performing time domain feature extraction on the differential entropy features based on a time domain attention network to obtain an EEG time domain feature matrix; performing spatial feature extraction on the EEG time domain feature matrix based on a channel attention network to obtain a target EEG feature matrix; performing geometric feature construction on the target EEG feature matrix based on a graph attention network to obtain an EEG feature graph structure; performing emotion recognition on the EEG feature graph structure based on a recognition network to obtain a target emotion label that represents the emotion category, and the target emotion label is used to represent the emotion category of the target EEG data. The embodiment of the present application can improve the accuracy of emotion recognition.

Description

Technical Field

[0001] The present application relates to the field of digital medical technology, and in particular to an emotion recognition method and device based on electroencephalogram (EEG) signals, an electronic device, and a medium. Background Art

[0002] Emotions are complex physiological responses. Positive emotions can improve health and work efficiency, while negative emotions can lead to health problems. Emotion recognition is widely used in a variety of scientific and technological fields, including human-computer interaction, distance education, and healthcare, and has garnered significant attention from the academic community.

[0003] In digital healthcare scenarios, a patient's emotional state often influences the treatment plans recommended by the consultation robot on the intelligent consultation platform. During the intelligent consultation process, the consultation robot uses emotion recognition based on physiological signals, including autonomic nervous system (ANS) and central nervous system (CNS) signals. The ANS method primarily uses heart rate, skin conductivity, respiration, and other factors to identify emotions. However, these studies lack reasonable evaluation criteria and accuracy. The CNS mainly uses electroencephalogram (EEG) signals to classify emotions. These signals are a direct reflection of brain activity and have good temporal resolution.

[0004] Most current emotion recognition methods are based on convolutional neural networks to extract local spatial features of the user's EEG signals and rely on the extracted local spatial features for emotion recognition. This approach often results in the loss of spatial feature information and the problem of low recognition accuracy. Therefore, how to improve the accuracy of emotion recognition has become a technical problem that needs to be solved urgently. Summary of the Invention

[0005] The main purpose of the embodiments of the present application is to propose an emotion recognition method and device based on EEG signals, an electronic device and a medium, aiming to improve the accuracy of emotion recognition.

[0006] To achieve the above objectives, a first aspect of an embodiment of the present application provides an emotion recognition method based on EEG signals, the method comprising:

[0007] Acquiring target EEG data of a target object, wherein the target EEG data includes a plurality of target EEG signals;

[0008] Performing frequency domain feature extraction on the target EEG signal to obtain differential entropy features;

[0009] Inputting the differential entropy feature into a preset emotion recognition model, wherein the emotion recognition model includes a temporal attention network, a channel attention network, a graph attention network, and a recognition network;

[0010] Performing time domain feature extraction on the differential entropy feature based on the time domain attention network to obtain an EEG time domain feature matrix;

[0011] Performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix;

[0012] Performing geometric feature construction on the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure;

[0013] Emotion recognition is performed on the EEG feature map structure based on the recognition network to obtain a target emotion label, where the target emotion label is used to characterize the emotion category of the target EEG data.

[0014] In some embodiments, extracting local information from the differential entropy feature based on the time-domain attention network to obtain an EEG time-domain feature matrix includes:

[0015] Performing an affine transformation on the differential entropy feature based on the self-attention layer of the time-domain attention network to obtain a time-domain query vector, a time-domain value vector, and a time-domain key vector of the target EEG data;

[0016] Performing weight configuration on the time-domain query vector, the time-domain value vector, and the time-domain key vector based on the first function of the time-domain attention network to obtain time-domain weight data;

[0017] The time domain query vector, the time domain value vector, and the time domain key vector are weighted and summed based on the time domain weight data to obtain the EEG time domain feature matrix.

[0018] In some embodiments, performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix includes:

[0019] Performing channel context extraction on the EEG time domain feature matrix based on the Bi-LSTM layer of the channel attention network to obtain an EEG channel feature matrix;

[0020] Obtaining spatial position data of the EEG channel feature matrix;

[0021] performing channel grouping on the EEG channel feature matrix based on the spatial position data to obtain a plurality of EEG channel groups, wherein each EEG channel group includes a plurality of the EEG channel feature matrices;

[0022] Performing channel attention calculation on each of the EEG channel groups based on the channel attention layer of the channel attention network to obtain a channel attention matrix for each EEG channel group;

[0023] Based on the channel attention matrix, the target EEG feature matrix is ​​obtained.

[0024] In some embodiments, the graph attention network includes an embedding layer and a graph attention layer, and the geometric feature construction of the target EEG feature matrix based on the graph attention network to obtain the EEG feature graph structure includes:

[0025] Performing graph embedding on the target EEG feature matrix based on the embedding layer to obtain EEG embedding features;

[0026] Performing geometric feature extraction on the EEG embedding features based on the graph attention layer to obtain the EEG structural feature matrix;

[0027] Based on the EEG structural feature matrix, the EEG feature map structure is constructed.

[0028] In some embodiments, the recognition network includes a pooling layer and a classifier, and performing emotion recognition on the EEG feature map structure based on the recognition network to obtain a target emotion label includes:

[0029] Performing sampling processing on the EEG feature map structure based on the pooling layer to obtain a target feature map structure;

[0030] Emotion recognition is performed on the target feature graph structure based on the classifier to obtain the target emotion label.

[0031] In some embodiments, performing emotion recognition on the target feature graph structure based on the classifier to obtain the target emotion label includes:

[0032] Performing an emotion score on the target feature graph structure based on the classifier and a preset candidate emotion category label to obtain an emotion score of the target EEG data on the candidate emotion category label;

[0033] Filtering the maximum emotion score from the emotion scores as the target score;

[0034] The target emotion label is filtered out from the candidate emotion category labels according to the target score.

[0035] In some embodiments, extracting frequency domain features from the target EEG signal to obtain differential entropy features includes:

[0036] Performing frame processing on the target EEG data to obtain target EEG signal frames;

[0037] Performing short-time Fourier transform on the target EEG signal frame to obtain the differential entropy feature.

[0038] To achieve the above-mentioned purpose, a second aspect of an embodiment of the present application provides an emotion recognition device based on EEG signals, the device comprising:

[0039] An EEG data acquisition module, configured to acquire target EEG data of a target subject, wherein the target EEG data includes a plurality of target EEG signals;

[0040] A signal feature extraction module is used to extract frequency domain features of the target EEG signal to obtain differential entropy features;

[0041] A feature input module, configured to input the differential entropy feature into a preset emotion recognition model, wherein the emotion recognition model includes a temporal attention network, a channel attention network, a graph attention network, and a recognition network;

[0042] A time domain feature extraction module, configured to perform time domain feature extraction on the differential entropy feature based on the time domain attention network to obtain an EEG time domain feature matrix;

[0043] A spatial feature extraction module is used to extract spatial features from the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix;

[0044] A graph structure generation module, configured to construct geometric features of the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure;

[0045] The emotion recognition module is used to perform emotion recognition on the EEG feature map structure based on the recognition network to obtain a target emotion label, and the target emotion label is used to characterize the emotion category of the target EEG data.

[0046] To achieve the above-mentioned purpose, the third aspect of an embodiment of the present application proposes an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the method described in the first aspect is implemented.

[0047] To achieve the above-mentioned purpose, the fourth aspect of the embodiments of the present application proposes a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the method described in the first aspect.

[0048] The present application proposes an EEG signal-based emotion recognition method, an EEG signal-based emotion recognition device, an electronic device, and a storage medium. The method obtains target EEG data of a target object, wherein the target EEG data includes multiple target EEG signals; extracts frequency domain features of the target EEG signals to obtain differential entropy features, and can use the differential entropy features containing high-quality EEG feature information to perform emotion recognition. Furthermore, the differential entropy features are input into a preset emotion recognition model, which includes a time domain attention network, a channel attention network, a graph attention network, and a recognition network; extracts time domain features of the differential entropy features based on the time domain attention network to obtain an EEG time domain feature matrix; extracts spatial features of the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix; and constructs geometric features of the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure. The method can more conveniently extract the features of the target EEG signal in the time domain and space domain, and obtain an EEG feature graph structure containing both time information and spatial information. At the same time, multiple attention mechanisms, such as temporal attention, channel attention, and graph attention, are introduced into the feature extraction process. This allows for better extraction of more important feature information from the target EEG signal for emotion recognition, reducing the interference of irrelevant redundant information on the emotion recognition process. Finally, emotion recognition is performed on the EEG feature graph structure based on the recognition network to obtain the target emotion label. The target emotion label is used to represent the emotion category of the target EEG data. This can more easily determine the emotion category represented by the target EEG data of the target subject, improving the accuracy of emotion recognition. This allows the consultation robot on the intelligent consultation platform to recommend more appropriate treatment plans based on the subject's emotional state, effectively improving the accuracy and flexibility of intelligent consultation and promoting the rapid development of medical cloud technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 This is a flow chart of an emotion recognition method based on EEG signals provided in an embodiment of the present application;

[0050] Figure 2 yes Figure 1 Flowchart of step S102 in FIG.

[0051] Figure 3 yes Figure 1 Flowchart of step S104 in FIG.

[0052] Figure 4 yes Figure 1 Flowchart of step S105 in FIG.

[0053] Figure 5 yes Figure 1 Flowchart of step S106 in FIG.

[0054] Figure 6 yes Figure 1 Flowchart of step S107 in FIG.

[0055] Figure 7 yes Figure 6 Flowchart of step S602 in FIG.

[0056] Figure 8 Schematic diagram of the structure of an emotion recognition device based on EEG signals provided in an embodiment of the present application;

[0057] Figure 9 This is a schematic diagram of the hardware structure of the electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0059] It should be noted that although the device schematics illustrate functional module divisions and the flowcharts illustrate logical sequences, in certain circumstances, the steps shown or described may be performed in a sequence that differs from the module divisions in the device or the sequence in the flowcharts. The terms "first," "second," and so on, in the specification, claims, and drawings, are used to distinguish similar items and are not necessarily used to describe a specific sequence or precedence.

[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this application pertains. The terms used herein are for the purpose of describing the embodiments of this application only and are not intended to limit this application.

[0061] First, let’s analyze some of the terms used in this application:

[0062] Artificial intelligence (AI) is a new technical discipline that studies and develops theories, methods, technologies, and application systems for simulating, extending, and expanding human intelligence. A branch of computer science, AI seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. Research in this field includes robotics, speech recognition, image recognition, natural language processing, and expert systems. AI can simulate the information processes of human consciousness and thinking. It also encompasses the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results.

[0063] Natural language processing (NLP): NLP uses computers to process, understand, and apply human languages ​​(such as Chinese and English). A branch of artificial intelligence, NLP is an interdisciplinary field between computer science and linguistics, often referred to as computational linguistics. Natural language processing encompasses grammatical analysis, semantic analysis, and discourse comprehension. Natural language processing is commonly used in technical fields such as machine translation, handwritten and printed character recognition, speech recognition and text-to-speech conversion, information intent recognition, information extraction and filtering, text classification and clustering, public opinion analysis, and opinion mining. It encompasses data mining, machine learning, knowledge acquisition, knowledge engineering, artificial intelligence research related to language processing, and linguistics research related to language computing.

[0064] Fourier transform: This function can be expressed as a linear combination of trigonometric functions (sine and / or cosine functions) or their integrals. In different research fields, the Fourier transform has many different variations, such as the continuous Fourier transform and the discrete Fourier transform.

[0065] Softmax function: The Softmax function is a normalized exponential function.

[0066] Emotions are complex physiological responses. Positive emotions can improve health and work efficiency, while negative emotions can lead to health problems. Emotion recognition is widely used in a variety of scientific and technological fields, including human-computer interaction, distance education, and healthcare, and has garnered significant attention from the academic community.

[0067] In digital healthcare scenarios, a patient's emotional state often influences the treatment plans recommended by the consultation robot on the intelligent consultation platform. During the intelligent consultation process, the consultation robot uses emotion recognition based on physiological signals, including autonomic nervous system (ANS) and central nervous system (CNS) signals. The ANS method primarily uses heart rate, skin conductivity, respiration, and other factors to identify emotions. However, these studies lack reasonable evaluation criteria and accuracy. The CNS mainly uses electroencephalogram (EEG) signals to classify emotions. These signals are a direct reflection of brain activity and have good temporal resolution.

[0068] Most current emotion recognition methods are based on convolutional neural networks to extract local spatial features of the user's EEG signals and rely on the extracted local spatial features for emotion recognition. This approach often results in the loss of spatial feature information and the problem of low recognition accuracy. Therefore, how to improve the accuracy of emotion recognition has become a technical problem that needs to be solved urgently.

[0069] Based on this, the embodiments of the present application provide an emotion recognition method based on EEG signals, an emotion recognition device based on EEG signals, an electronic device and a storage medium, aiming to improve the accuracy of emotion recognition.

[0070] The EEG signal-based emotion recognition method, EEG signal-based emotion recognition device, electronic device and storage medium provided in the embodiments of the present application are specifically illustrated through the following embodiments. First, the emotion recognition method in the embodiments of the present application is described.

[0071] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Artificial Intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to achieve optimal results.

[0072] Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interaction systems, and mechatronics. AI software technologies primarily encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0073] The method for emotion recognition based on EEG signals provided in the embodiment of the present application relates to the field of artificial intelligence technology. The method for emotion recognition based on EEG signals provided in the embodiment of the present application can be applied in a terminal, can be applied in a server side, or can be software running in a terminal or a server side. In some embodiments, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.; the server side can be configured as an independent physical server, or as a server cluster or distributed system composed of multiple physical servers, or as a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms; the software can be an application that implements the method for emotion recognition based on EEG signals, etc., but is not limited to the above forms.

[0074] Medical cloud refers to the use of cloud computing to create a cloud-based healthcare service platform based on new technologies such as cloud computing, mobile technology, multimedia, 4G communications, big data, and the Internet of Things, combined with medical technologies. This platform enables the sharing of medical resources and expands the scope of care. By integrating cloud computing technologies, medical cloud improves the efficiency of medical institutions and facilitates access to healthcare for residents. Current hospital appointment systems, electronic medical records, and medical insurance are all products of the integration of cloud computing and the healthcare sector. Medical cloud also offers advantages such as data security, information sharing, dynamic scalability, and a comprehensive approach.

[0075] It should be noted that in each specific embodiment of the present application, when it comes to the need to perform relevant processing based on data related to the user's identity or characteristics, such as user information, user behavior data, user historical data, and user location information, the user's permission or consent will be obtained first, and the collection, use, and processing of such data will comply with relevant laws, regulations, and standards. In addition, when the embodiment of the present application needs to obtain the user's sensitive personal information, the user's separate permission or consent will be obtained through a pop-up window or by jumping to a confirmation page. After clearly obtaining the user's separate permission or consent, the necessary user-related data for the normal operation of the embodiment of the present application will be obtained.

[0076] The present application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like. The present application can be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The present application can also be practiced in distributed computing environments in which tasks are performed by remote processing devices connected via a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0077] Figure 1 This is an optional flowchart of the method for emotion recognition based on EEG signals provided in an embodiment of the present application. Figure 1 The method may include but is not limited to steps S101 to S107.

[0078] Step S101, acquiring target EEG data of a target object, wherein the target EEG data includes a plurality of target EEG signals;

[0079] Step S102, extracting frequency domain features of the target EEG signal to obtain differential entropy features;

[0080] Step S103: inputting the differential entropy features into a preset emotion recognition model, where the emotion recognition model includes a temporal attention network, a channel attention network, a graph attention network, and a recognition network;

[0081] Step S104, performing time domain feature extraction on the differential entropy feature based on the time domain attention network to obtain an EEG time domain feature matrix;

[0082] Step S105, performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix;

[0083] Step S106, constructing geometric features of the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure;

[0084] Step S107 , performing emotion recognition on the EEG feature map structure based on the recognition network to obtain a target emotion label, which is used to represent the emotion category of the target EEG data.

[0085] In the embodiment of the present application, steps S101 to S107 are performed by obtaining target EEG data of the target object, wherein the target EEG data includes multiple target EEG signals; performing frequency domain feature extraction on the target EEG signals to obtain differential entropy features, and being able to use the differential entropy features containing high-quality EEG feature information for emotion recognition. Further, the differential entropy features are input into a preset emotion recognition model, which includes a time domain attention network, a channel attention network, a graph attention network, and a recognition network; performing time domain feature extraction on the differential entropy features based on the time domain attention network to obtain an EEG time domain feature matrix; performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix, and performing geometric feature construction on the target EEG feature matrix based on the graph attention network to obtain an EEG feature map structure, which can more conveniently extract the features of the target EEG signal in the time domain and space domains, and obtain an EEG feature map structure containing both time information and spatial information. At the same time, the feature extraction process incorporates multiple attention mechanisms, including temporal attention, channel attention, and graph attention. This effectively extracts more important features from the target EEG signal for emotion recognition, reducing the interference of irrelevant redundant information on the emotion recognition process. Finally, emotion recognition is performed on the EEG feature graph structure based on a recognition network to obtain a target emotion label. This target emotion label is used to represent the emotion category of the target EEG data. This makes it easier to determine the emotion category represented by the target EEG data, improving the accuracy of emotion recognition.

[0086] In step S101 of some embodiments, the target EEG data of the target object can be extracted from a preset database, or the target EEG data of the target object can be obtained by writing a web crawler, setting up the data source, and then crawling the data in a targeted manner, wherein the data source can be various types of network platforms, commonly used EEG signal emotion recognition data sets (such as SEED data sets), etc. In the digital medical scenario, the target object can be a patient, etc.; in the field of recommendation or other interactive services, the target object can be a consumer, a network user, etc., without limitation. The target EEG data contains EEG signals of multiple EEG channels, that is, the target EEG data includes multiple target EEG signals, and different target EEG signals contain different physiological characteristic information of the target object. The above process can make the acquired target EEG data richer and more comprehensive, which is conducive to improving the accuracy of emotion recognition.

[0087] See also Figure 2 In some embodiments, step S102 may include but is not limited to steps S201 to S202:

[0088] Step S201, performing frame processing on the target EEG data to obtain target EEG signal frames;

[0089] Step S202 , performing short-time Fourier transform on the target EEG signal frame to obtain differential entropy features.

[0090] In step S201 of some embodiments, when frequency domain features are extracted from the target EEG data, each target EEG signal is first framed and windowed, and the target EEG signal is split into frames according to the signal duration of the target EEG signal of each EEG channel to obtain multiple target EEG signal frames.

[0091] In step S202 of some embodiments, a short-time Fourier transform is performed on each target EEG signal frame to convert the time domain features of the target EEG signal frame into frequency domain features, thereby obtaining a differential entropy feature corresponding to each target EEG signal. The differential entropy feature is an EEG feature in the target EEG signal that can better reflect the emotional changes of the target object, and the differential entropy feature is a nonlinear feature.

[0092] Through the above steps S201 to S202, the target EEG data can be converted into spectral features more conveniently, so that the emotion category of the target object can be identified through the differential entropy features, thereby improving the accuracy of emotion recognition.

[0093] In step S103 of some embodiments, the differential entropy features can be input into a preset emotion recognition model using a preset script program. The emotion recognition model can be constructed based on multiple attention mechanisms. The emotion recognition model includes a time domain attention network, a channel attention network, a graph attention network and a recognition network. The time domain attention network is mainly used to extract local information of the input EEG features in the time domain across channels to obtain the EEG time domain feature matrix; the channel attention network is mainly used to use a bidirectional long short-term memory algorithm to learn the high-level feature representation in the EEG time domain feature matrix to obtain the target EEG feature matrix; the graph attention network is mainly used to perform graph embedding construction on the target EEG feature matrix to form an EEG feature graph structure; the recognition network is mainly used to use a preset classifier to perform type recognition on the EEG feature graph structure to determine the emotion category corresponding to the input EEG feature. Since EEG signals have a three-dimensional topological structure, the emotion recognition model can realize feature extraction of multiple dimensions such as the time domain and spatial domain of the target EEG data, and can capture local time information at different times and spatial information between the same channel (i.e., within the same brain area) and different channels (i.e., between different brain areas). It enables the use of EEG features containing relatively complete spatiotemporal information for emotion recognition, thereby improving the accuracy of emotion recognition.

[0094] See also Figure 3 In some embodiments, step S104 may include but is not limited to steps S301 to S303:

[0095] Step S301: performing an affine transformation on the differential entropy feature based on the self-attention layer of the time-domain attention network to obtain the time-domain query vector, time-domain value vector, and time-domain key vector of the target EEG data;

[0096] Step S302, weighting the time-domain query vector, the time-domain value vector, and the time-domain key vector based on the first function of the time-domain attention network to obtain time-domain weight data;

[0097] Step S303 , performing weighted summation on the time domain query vector, the time domain value vector, and the time domain key vector based on the time domain weight data to obtain an EEG time domain feature matrix.

[0098] In step S301 of some embodiments, the temporal attention network includes multiple self-attention layers, and an affine transformation is performed on a differential entropy feature based on a self-attention layer of the temporal attention network, and the differential entropy feature a is multiplied by a preset affine coefficient W. q1 、W k1 、W v1 , respectively obtain the time domain query vector Q1, time domain value vector V1 and time domain key vector K1 of the differential entropy feature a, where the time domain query vector Q1 can be expressed as Q1 = a·W q1, the time domain value vector can be expressed as V1 = a·W v1 , the time domain key vector can be expressed as K1 = a·W k1 .

[0099] In step S302 of some embodiments, the first function may be a cosine similarity function, a softmax function, or other probability functions, without limitation. Taking the softmax function as an example, the softmax function is used to perform the transposition K1 of the time domain key vector K1. T The dot product matrix between the time domain query vector Q1 is activated to obtain the time domain weight data, that is, the time domain weight data can be expressed as M = softmax (A), where A = K1 T Q1, where T represents the transpose operation on the time-domain key vector.

[0100] In step S303 of some embodiments, when weighted summing the time domain query vector, the time domain value vector and the time domain key vector is performed based on the time domain weight data, the time domain weight data and the time domain value vector can be directly multiplied to obtain the EEG time domain feature matrix N corresponding to a certain target EEG signal, where the EEG time domain feature matrix can be expressed as N=V1·M.

[0101] Through the above steps S301 to S303, the feature information of the time domain dimension of the target EEG data can be extracted, the local time information of the target EEG data at different moments can be captured, and the feature richness of the EEG time domain feature matrix is ​​improved.

[0102] See also Figure 4 In some embodiments, step S105 may include but is not limited to steps S401 to S405:

[0103] Step S401: performing channel context extraction on the EEG time domain feature matrix based on the Bi-LSTM layer of the channel attention network to obtain the EEG channel feature matrix;

[0104] Step S402, obtaining spatial position data of the EEG channel feature matrix;

[0105] Step S403, grouping the EEG channel feature matrix based on the spatial position data to obtain a plurality of EEG channel groups, wherein each EEG channel group includes a plurality of EEG channel feature matrices;

[0106] Step S404: performing channel attention calculation on each EEG channel group based on the channel attention layer of the channel attention network to obtain a channel attention matrix for each EEG channel group;

[0107] Step S405: Obtain a target EEG feature matrix based on the channel attention matrix.

[0108] In step S401 of some embodiments, the channel attention network includes multiple Bi-LSTM layers, and a Bi-LSTM layer performs spatial feature extraction on an EEG time domain feature matrix. When the Bi-LSTM layer based on the channel attention network performs channel context extraction on the EEG time domain feature matrix, a certain EEG time domain feature matrix is ​​forward encoded from the time dimension at a certain Bi-LSTM layer to obtain a forward encoding latent vector at each moment, and then the EEG time domain feature matrix is ​​reversely encoded from the time dimension at the Bi-LSTM layer to obtain a reverse encoding latent vector at each moment, and the forward encoding latent vector and the reverse encoding latent vector at each moment are vector-concatenated to obtain a channel encoding vector at each moment, and the channel encoding vectors at all moments are aggregated into a matrix to obtain an EEG channel feature matrix corresponding to a target EEG signal (i.e., a certain EEG channel).

[0109] In step S402 of some embodiments, the EEG channel feature matrices of all EEG channels can be spatially modeled using a preset spatial coordinate system to obtain spatial position data of each EEG channel feature matrix. The spatial position data includes the coordinate parameters of each EEG channel feature matrix. Information such as the adjacent position relationship between EEG channels can also be clearly determined from the spatial position data.

[0110] In step S403 of some embodiments, since the responses of neurons to different emotional stimuli may vary in different areas of the cerebral cortex, in order to improve the accuracy of emotion recognition, the EEG channels may be divided into multiple EEG channel groups during the emotion recognition process to extract spatial features of the EEG signals. For example, based on the adjacent positional relationship between the EEG channels in the spatial position data, the EEG channel feature matrix may be grouped to obtain multiple EEG channel groups, wherein each EEG channel group contains multiple EEG channel feature matrices. For example, in a certain emotion recognition scenario, there are 8 EEG channels, each of which has a corresponding EEG channel feature matrix. According to actual needs, the 8 EEG channels are divided into two groups. Then, according to the positional relationship of the EEG channels, the first 4 EEG channels are divided into one group, denoted as EEG channel group A, and the last 4 EEG channels are divided into one group, denoted as EEG channel group B. wherein EEG channel group A contains the 4 EEG channel feature matrices corresponding to the first 4 EEG channels, and EEG channel group B contains the 4 EEG channel feature matrices corresponding to the last 4 EEG channels.

[0111] It should be noted that when the EEG channel feature matrix is ​​grouped, the number of EEG channels and EEG channel feature matrices in each EEG channel group can be the same or different, without limitation.

[0112] In step S404 of some embodiments, when the channel attention layer based on the channel attention network performs channel attention calculation on each EEG channel group, all EEG channel feature matrices of each EEG channel group are first merged to obtain a fused channel feature matrix of each EEG channel group, and then the channel key matrix, channel value matrix and channel query matrix of each fused channel feature matrix P are calculated respectively. The channel key matrix, channel value matrix and channel query matrix are weighted by the softmax function to obtain the channel attention matrix of each EEG channel group, where the channel key matrix can be expressed as K2=P*W k2 , the channel value matrix can be expressed as V2=P*W v2 , the channel query matrix can be expressed as Q2=P*W q2 , where P is the fusion channel feature matrix, W k2 、W v2 、W q2 is a preset parameter. The channel attention matrix Z can be expressed as shown in formula (1), where d is the feature dimension of the fused channel feature matrix, and T represents the transposition operation of the channel key matrix K2:

[0113]

[0114] Through the above attention calculation process, the mapping of important feature information in the fusion channel feature matrix can be strengthened, while the mapping of minor feature information in the fusion channel feature matrix can be reduced.

[0115] In step S405 of some embodiments, all channel attention matrices may be directly merged to obtain a target EEG feature matrix, which contains spatiotemporal feature information of the target EEG data in the time domain and the spatial domain.

[0116] Through the above steps S401 to S405, the feature extraction of the spatial domain dimension of the target EEG data can be achieved, and the spatial feature information of the target EEG data in the same channel (i.e., within the same brain area) and between different channels (i.e., between different brain areas) can be captured. The spatial feature information of the target EEG data can be used for emotion recognition, which can effectively improve the accuracy of emotion recognition.

[0117] See also Figure 5 In some embodiments, the graph attention network includes an embedding layer and a graph attention layer. Step S106 may include but is not limited to steps S501 to S503:

[0118] Step S501, performing graph embedding on the target EEG feature matrix based on the embedding layer to obtain EEG embedding features;

[0119] Step S502: extract geometric features from the EEG embedding features based on the graph attention layer to obtain an EEG structural feature matrix;

[0120] Step S503: constructing an EEG feature map structure based on the EEG structural feature matrix.

[0121] In step S501 of some embodiments, the target EEG feature matrix can first be subjected to graph embedding processing based on the embedding layer, and the affine transformation can be applied to the eigenvector of each EEG channel group in the target EEG feature matrix. The target EEG feature matrix that has undergone the affine transformation is then subjected to shared linear mapping, and the target EEG feature matrix that has undergone the affine transformation is mapped to a fixed geometric space, thereby obtaining the graph embedding representation corresponding to the target EEG data and obtaining the EEG embedding feature.

[0122] In step S502 of some embodiments, when geometric features are extracted from EEG embedded features based on the graph attention layer, a multi-head attention mechanism can be introduced to extract features from the EEG embedded features. Specifically, multiple attention heads are used to perform shared attention mapping, and each attention head is used to perform attention calculation on the EEG embedded features to obtain an attention coefficient. The importance of EEG features between different attention heads is determined based on the size of the attention coefficient. Based on different attention coefficients, different feature weights are assigned to EEG feature information of different geometric regions in the EEG embedded features. The EEG embedded features are reconstructed based on the feature weights to obtain an EEG structural feature matrix.

[0123] It should be noted that in order to improve computational efficiency, when using each attention head to perform attention calculation on the EEG embedding features and obtain the attention coefficient, a parallel computing graph attention mechanism can be used to implement it. For example, K independent attention operators are used to calculate the attention coefficient in parallel, where K is an integer greater than zero, thereby saving computing time and improving the efficiency of emotion recognition.

[0124] In step S503 of some embodiments, a preset format conversion tool is used to convert the EEG structural feature matrix from a matrix form to a graph structure to obtain an EEG feature graph structure. The EEG feature graph structure can be a two-dimensional graph structure or a three-dimensional graph structure, and the EEG feature graph structure includes regional emotion representation information for each EEG channel.

[0125] Through the above steps S501 to S503, the target EEG feature matrix containing spatiotemporal feature information can be converted into a graph structure more conveniently, and emotion recognition is performed using the EEG feature graph structure. After weight allocation and feature reconstruction operations, the obtained EEG feature graph structure contains EEG features in the target EEG signal that are highly correlated with the emotional content, and has less irrelevant redundant information, which can better improve the accuracy of emotion recognition.

[0126] See also Figure 6 In some embodiments, the recognition network includes a pooling layer and a classifier, and step S107 includes but is not limited to steps S601 to S602:

[0127] Step S601: Sampling the EEG feature map structure based on the pooling layer to obtain a target feature map structure;

[0128] Step S602: emotion recognition is performed on the target feature graph structure based on the classifier to obtain a target emotion label.

[0129] In step S601 of some embodiments, when sampling and processing the EEG feature map structure based on the pooling layer, an average pooling method can be adopted. For example, the regional emotion representation features of each EEG channel of each EEG channel group in the EEG feature map structure are extracted, the regional emotion representation features of each EEG channel group are averaged to obtain the global emotion representation features of each EEG channel group, and all global emotion representation features are graph-structured to obtain the target feature map structure.

[0130] In step S602 of some embodiments, the classifier can be a softmax classifier, etc., without limitation. Taking the softmax classifier as an example, a probability distribution of the target feature graph structure on each candidate emotion category label is created based on the softmax classifier to achieve an emotion score for the target feature graph structure, and the probability distribution vector of each candidate emotion category label is used as the emotion score of the target EEG data on the candidate emotion category label. Since the size of the emotion score can directly reflect the possibility that the target EEG data belongs to each candidate emotion category label, that is, the larger the emotion score, the more the emotion feature in the target EEG data is biased towards the candidate emotion category corresponding to the emotion score. Therefore, the maximum emotion score can be screened out from the emotion score as the target score, and the candidate emotion category label corresponding to the target score can be used as the target emotion label, which can characterize the emotion category represented by the target EEG data of the target object.

[0131] It should be noted that the candidate emotion category labels can be set based on expert experience, etc., and the candidate emotion category labels can include multiple emotion categories such as joy, anger, sadness, happiness, etc., without restriction.

[0132] Through the above steps S601 to S602, the emotion category of the target object can be clearly judged based on the target EEG data of the target object, and the probability distribution size of the target EEG data on each emotion label can be predicted based on the target feature graph structure corresponding to the target EEG data, which can better improve the accuracy of emotion recognition of the target object.

[0133] See also Figure 7 In some embodiments, step S602 may include but is not limited to steps S701 to S703:

[0134] Step S701: Emotionally score the target feature graph structure based on the classifier and the preset candidate emotion category label to obtain the emotion score of the target EEG data on the candidate emotion category label;

[0135] Step S702, filtering out the maximum emotion score from the emotion scores as the target score;

[0136] Step S703: Filter out a target emotion label from the candidate emotion category labels according to the target score.

[0137] In step S701 of some embodiments, the classifier may be a softmax classifier, etc., without limitation. Taking the softmax classifier as an example, a probability distribution of the target feature graph structure on each candidate emotion category label is created based on the softmax classifier to implement an emotion score for the target feature graph structure. The probability distribution vector of each candidate emotion category label is used as the emotion score for the target EEG data on the candidate emotion category label.

[0138] In step S702 of some embodiments, since the magnitude of the emotion score directly reflects the likelihood that the target EEG data belongs to each candidate emotion category label, that is, the larger the emotion score, the more likely the emotion features in the target EEG data are to belong to the candidate emotion category corresponding to the emotion score. Therefore, the maximum emotion score can be selected from the emotion scores as the target score.

[0139] In step S703 of some embodiments, after determining the target score, the candidate emotion category labels are screened according to the target score, and the candidate emotion category label corresponding to the target score is selected as the target emotion label, which can be used to characterize the emotion category of the target EEG data.

[0140] Through the above steps S701 to S703, the emotion category of the target object can be clearly judged based on the target EEG data of the target object, and the probability distribution size of the target EEG data on each emotion label can be predicted based on the target feature graph structure corresponding to the target EEG data. The emotion score is used to quantify the possibility that the emotion information represented in the target EEG data belongs to each candidate emotion category label, which can greatly improve the accuracy of emotion recognition.

[0141] The emotion recognition method based on EEG signals of the embodiment of the present application obtains the target EEG data of the target object, wherein the target EEG data includes multiple target EEG signals; extracts the frequency domain features of the target EEG signals to obtain differential entropy features, and can use the differential entropy features containing high-quality EEG feature information to perform emotion recognition. Further, the differential entropy features are input into a preset emotion recognition model, and the emotion recognition model includes a time domain attention network, a channel attention network, a graph attention network and a recognition network; based on the time domain attention network, the differential entropy features are extracted in the time domain to obtain an EEG time domain feature matrix; based on the channel attention network, the EEG time domain feature matrix is ​​extracted in the spatial domain to obtain a target EEG feature matrix, and based on the graph attention network, the target EEG feature matrix is ​​geometrically constructed to obtain an EEG feature map structure, which can more conveniently extract the features of the target EEG signal in the time domain and space domain, and obtain an EEG feature map structure containing both time information and spatial information. At the same time, multiple attention mechanisms, such as temporal attention, channel attention, and graph attention, are introduced into the feature extraction process. This allows for better extraction of more important feature information from the target EEG signal for emotion recognition, reducing the interference of irrelevant redundant information on the emotion recognition process. Finally, emotion recognition is performed on the EEG feature graph structure based on the recognition network to obtain the target emotion label. The target emotion label is used to represent the emotion category of the target EEG data. This can more easily determine the emotion category represented by the target EEG data of the target subject, improving the accuracy of emotion recognition. This allows the consultation robot on the intelligent consultation platform to recommend more appropriate treatment plans based on the subject's emotional state, effectively improving the accuracy and flexibility of intelligent consultation and promoting the rapid development of medical cloud technology.

[0142] See also Figure 8 The present application also provides an embodiment of an emotion recognition device based on EEG signals, which can implement the above-mentioned emotion recognition method based on EEG signals. The device includes:

[0143] The EEG data acquisition module 801 is used to acquire target EEG data of a target object, wherein the target EEG data includes multiple target EEG signals;

[0144] The signal feature extraction module 802 is used to extract the frequency domain features of the target EEG signal to obtain differential entropy features;

[0145] A feature input module 803 is used to input the differential entropy feature into a preset emotion recognition model, which includes a temporal attention network, a channel attention network, a graph attention network, and a recognition network;

[0146] The time domain feature extraction module 804 is used to extract the time domain features of the differential entropy features based on the time domain attention network to obtain the EEG time domain feature matrix;

[0147] The spatial feature extraction module 805 is used to extract spatial features from the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix;

[0148] A graph structure generation module 806 is used to construct geometric features of the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure;

[0149] The emotion recognition module 807 is used to perform emotion recognition on the EEG feature map structure based on the recognition network to obtain a target emotion label, which is used to represent the emotion category of the target EEG data.

[0150] The specific implementation of the emotion recognition device based on EEG signals is basically the same as the specific embodiment of the above-mentioned emotion recognition method based on EEG signals, and will not be repeated here.

[0151] The present application also provides an electronic device comprising: a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for enabling communication between the processor and the memory. When the program is executed by the processor, the aforementioned method for emotion recognition based on electroencephalogram signals is implemented. The electronic device can be any intelligent terminal, including a tablet computer and an in-vehicle computer.

[0152] See also Figure 9 , Figure 9 The hardware structure of an electronic device according to another embodiment is shown. The electronic device includes:

[0153] The processor 901 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present application;

[0154] The memory 902 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 902 can store an operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented through software or firmware, the relevant program codes are stored in the memory 902 and are called by the processor 901 to execute the method for emotion recognition based on EEG signals in the embodiments of this application.

[0155] Input / output interface 903, used to implement information input and output;

[0156] Communication interface 904, used to implement communication interaction between this device and other devices, which can be achieved through wired means (such as USB, network cable, etc.) or wireless means (such as mobile network, WiFi, Bluetooth, etc.);

[0157] Bus 905 , which transmits information between various components of the device (e.g., processor 901 , memory 902 , input / output interface 903 , and communication interface 904 );

[0158] The processor 901 , the memory 902 , the input / output interface 903 and the communication interface 904 are connected to each other in communication within the device via a bus 905 .

[0159] An embodiment of the present application also provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the above-mentioned emotion recognition method based on EEG signals.

[0160] The memory, as a non-transient computer-readable storage medium, can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] The embodiment of the present application provides an EEG signal-based emotion recognition method, an EEG signal-based emotion recognition device, an electronic device, and a computer-readable storage medium. The method obtains target EEG data of a target object, wherein the target EEG data includes multiple target EEG signals; extracts frequency domain features of the target EEG signals to obtain differential entropy features, and can use the differential entropy features containing high-quality EEG feature information to perform emotion recognition. Furthermore, the differential entropy features are input into a preset emotion recognition model, which includes a time domain attention network, a channel attention network, a graph attention network, and a recognition network; extracts time domain features of the differential entropy features based on the time domain attention network to obtain an EEG time domain feature matrix; extracts spatial features of the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix; and constructs geometric features of the target EEG feature matrix based on the graph attention network to obtain an EEG feature map structure. The method can more conveniently extract the features of the target EEG signal in the time domain and space domain, and obtain an EEG feature map structure containing both time information and spatial information. At the same time, multiple attention mechanisms, such as temporal attention, channel attention, and graph attention, are introduced into the feature extraction process. This allows for better extraction of more important feature information from the target EEG signal for emotion recognition, reducing the interference of irrelevant redundant information on the emotion recognition process. Finally, emotion recognition is performed on the EEG feature graph structure based on the recognition network to obtain the target emotion label. The target emotion label is used to represent the emotion category of the target EEG data. This can more easily determine the emotion category represented by the target EEG data of the target subject, improving the accuracy of emotion recognition. This allows the consultation robot on the intelligent consultation platform to recommend more appropriate treatment plans based on the subject's emotional state, effectively improving the accuracy and flexibility of intelligent consultation and promoting the rapid development of medical cloud technology.

[0162] The embodiments described in the embodiments of this application are intended to more clearly illustrate the technical solutions of the embodiments of this application and do not constitute a limitation on the technical solutions provided by the embodiments of this application. Those skilled in the art will appreciate that with the evolution of technology and the emergence of new application scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0163] It will be understood by those skilled in the art that Figure 1-7 The technical solutions shown in the figures do not constitute a limitation on the embodiments of the present application, and may include more or fewer steps than those shown in the figures, or a combination of certain steps, or different steps.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, i.e., they may be located in one place or distributed across multiple network units. Some or all of the modules may be selected based on actual needs to achieve the objectives of this embodiment.

[0165] Those skilled in the art will appreciate that all or some of the steps in the methods, systems, and functional modules / units in the devices disclosed above may be implemented as software, firmware, hardware, or appropriate combinations thereof.

[0166] The terms "first", "second", "third", "fourth", etc. (if any) in the specification of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0167] It should be understood that in this application, "at least one (item)" means one or more, and "plurality" means two or more. "And / or" is used to describe the association relationship of associated objects, indicating that three relationships may exist. For example, "A and / or B" can mean: only A exists, only B exists, and A and B exist at the same time, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following items" or similar expressions refers to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, c can be single or multiple.

[0168] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0169] The units described above as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0170] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0171] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes multiple instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned storage medium includes: various media that can store programs, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0172] The preferred embodiments of the present invention are described above with reference to the accompanying drawings, but are not intended to limit the scope of the present invention. Any modifications, equivalent substitutions, and improvements made by those skilled in the art without departing from the scope and essence of the present invention should be within the scope of the present invention.

Claims

1. An emotion recognition method based on EEG signals, characterized in that: The method comprises: Acquiring target EEG data of a target object, wherein the target EEG data includes a plurality of target EEG signals; Performing frequency domain feature extraction on the target EEG signal to obtain differential entropy features; inputting the differential entropy features into a preset emotion recognition model, wherein the emotion recognition model includes a time domain attention network, a channel attention network, a graph attention network, and a recognition network; Performing time domain feature extraction on the differential entropy feature based on the time domain attention network to obtain an EEG time domain feature matrix; performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix; Based on the graph attention network, geometric features of the target EEG feature matrix are constructed to obtain an EEG feature graph structure; based on the recognition network, emotion recognition is performed on the EEG feature graph structure to obtain a target emotion label, and the target emotion label is used to characterize the emotion category of the target EEG data; The time domain feature extraction of the differential entropy feature based on the time domain attention network to obtain the EEG time domain feature matrix includes: Performing an affine transformation on the differential entropy feature based on the self-attention layer of the time-domain attention network to obtain a time-domain query vector, a time-domain value vector, and a time-domain key vector of the target EEG data; performing weight configuration on the time-domain query vector, the time-domain value vector, and the time-domain key vector based on the first function of the time-domain attention network to obtain time-domain weight data; performing a weighted summation on the time-domain query vector, the time-domain value vector, and the time-domain key vector based on the time-domain weight data to obtain the EEG time-domain feature matrix; The step of performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix includes: Based on the Bi-LSTM layer of the channel attention network, channel context extraction is performed on the EEG time domain feature matrix to obtain an EEG channel feature matrix; spatial position data of the EEG channel feature matrix is ​​obtained; based on the spatial position data, channel grouping is performed on the EEG channel feature matrix to obtain multiple EEG channel groups, wherein each EEG channel group includes multiple EEG channel feature matrices; based on the channel attention layer of the channel attention network, channel attention calculation is performed on each of the EEG channel groups to obtain a channel attention matrix for each EEG channel group; based on the channel attention matrix, the target EEG feature matrix is ​​obtained.

2. The emotion recognition method according to claim 1, characterized in that The graph attention network includes an embedding layer and a graph attention layer. The geometric feature construction of the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure includes: Performing graph embedding on the target EEG feature matrix based on the embedding layer to obtain EEG embedding features; Performing geometric feature extraction on the EEG embedding features based on the graph attention layer to obtain an EEG structural feature matrix; Based on the EEG structural feature matrix, the EEG feature map structure is constructed.

3. The emotion recognition method according to claim 1, characterized in that The recognition network includes a pooling layer and a classifier. The emotion recognition is performed on the EEG feature map structure based on the recognition network to obtain a target emotion label, including: Performing sampling processing on the EEG feature map structure based on the pooling layer to obtain a target feature map structure; Emotion recognition is performed on the target feature graph structure based on the classifier to obtain the target emotion label.

4. The emotion recognition method according to claim 3, characterized in that The performing emotion recognition on the target feature graph structure based on the classifier to obtain the target emotion label includes: Performing an emotion score on the target feature graph structure based on the classifier and a preset candidate emotion category label to obtain an emotion score of the target EEG data on the candidate emotion category label; Filtering the maximum emotion score from the emotion scores as the target score; The target emotion label is filtered out from the candidate emotion category labels according to the target score.

5. The emotion recognition method according to any one of claims 1 to 4, characterized in that: The frequency domain feature extraction of the target EEG signal to obtain differential entropy features includes: Performing frame processing on the target EEG data to obtain target EEG signal frames; Performing short-time Fourier transform on the target EEG signal frame to obtain the differential entropy feature.

6. An emotion recognition device based on EEG signals, characterized in that: The device comprises: An EEG data acquisition module, configured to acquire target EEG data of a target subject, wherein the target EEG data includes a plurality of target EEG signals; A signal feature extraction module is used to extract frequency domain features of the target EEG signal to obtain differential entropy features; A feature input module, configured to input the differential entropy feature into a preset emotion recognition model, wherein the emotion recognition model includes a temporal attention network, a channel attention network, a graph attention network, and a recognition network; A time domain feature extraction module, configured to perform time domain feature extraction on the differential entropy feature based on the time domain attention network to obtain an EEG time domain feature matrix; A spatial feature extraction module is used to extract spatial features from the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix; A graph structure generation module, configured to construct geometric features of the target EEG feature matrix based on the graph attention network to obtain an EEG feature graph structure; An emotion recognition module, configured to perform emotion recognition on the EEG feature map structure based on the recognition network to obtain a target emotion label, wherein the target emotion label is used to characterize the emotion category of the target EEG data; The time domain feature extraction of the differential entropy feature based on the time domain attention network to obtain the EEG time domain feature matrix includes: Performing an affine transformation on the differential entropy feature based on the self-attention layer of the time-domain attention network to obtain a time-domain query vector, a time-domain value vector, and a time-domain key vector of the target EEG data; performing weight configuration on the time-domain query vector, the time-domain value vector, and the time-domain key vector based on the first function of the time-domain attention network to obtain time-domain weight data; performing a weighted summation on the time-domain query vector, the time-domain value vector, and the time-domain key vector based on the time-domain weight data to obtain the EEG time-domain feature matrix; The step of performing spatial feature extraction on the EEG time domain feature matrix based on the channel attention network to obtain a target EEG feature matrix includes: Based on the Bi-LSTM layer of the channel attention network, channel context extraction is performed on the EEG time domain feature matrix to obtain an EEG channel feature matrix; spatial position data of the EEG channel feature matrix is ​​obtained; based on the spatial position data, channel grouping is performed on the EEG channel feature matrix to obtain multiple EEG channel groups, wherein each EEG channel group includes multiple EEG channel feature matrices; based on the channel attention layer of the channel attention network, channel attention calculation is performed on each of the EEG channel groups to obtain a channel attention matrix for each EEG channel group; based on the channel attention matrix, the target EEG feature matrix is ​​obtained.

7. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores a computer program, and the processor implements the emotion recognition method based on EEG signals as described in any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for emotion recognition based on electroencephalogram signals according to any one of claims 1 to 5 is implemented.

Images