An emotion recognition method, device, equipment and storage medium

Abstract

The present invention discloses an emotion recognition method, device, equipment and storage medium. The method includes: obtaining a training data set and performing feature extraction processing on the sample data in the training data set; inputting the processed sample data and the emotion labels corresponding to the sample data into an attention recurrent convolutional network for model training to obtain an emotion recognition model; wherein, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit and an attention mechanism; inputting the target electroencephalogram signal to be recognized into the emotion recognition model, and taking the output result of the emotion recognition model as the emotion recognition result of the target electroencephalogram signal. The technical solution of the present invention has the beneficial effects of constructing a new emotion recognition model and improving the efficiency and accuracy of emotion recognition based on electroencephalogram signals.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular, to an emotion recognition method, device, equipment and storage medium. Background Art

[0002] Emotion is one of the advanced functions of human beings. Usually, anyone can express a variety of emotions. In terms of emotion recognition, through deep learning algorithms, enabling a computer to analyze the change of emotion by collecting various physiological signals when a person's mood changes is the most accurate among all emotion recognition methods.

[0003] At present, the mainstream algorithms of deep learning include Convolutional Neural Network (CNN) and Recurrent Neural Network (RNN). Since physiological signals have temporality and there is a certain correlation between front and back signals, and CNN cannot effectively interpret the time attribute when processing data with time characteristics, so RNN is usually used for calculation. However, the calculation amount of RNN is large, the model training time is long, and it is easy to reach local optimum, and it cannot meet the needs of emotion recognition in terms of calculation efficiency. Summary of the Invention

[0004] The present invention provides an emotion recognition method, device, equipment and storage medium to solve the problem of low efficiency and accuracy of emotion recognition of time series networks such as RNN.

[0005] According to one aspect of the present invention, there is provided an emotion recognition method, including:

[0006] Obtain a training data set, and perform feature extraction processing on the sample data in the training data set;

[0007] Input the processed sample data and the emotion label corresponding to the sample data into an attention recurrent convolutional network for model training to obtain an emotion recognition model;

[0008] Wherein, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit and an attention mechanism;

[0009] Input the target electroencephalogram signal to be recognized into the emotion recognition model, and use the output result of the emotion recognition model as the emotion recognition result of the target electroencephalogram signal.

[0010] According to another aspect of the present invention, there is provided an emotion recognition device, including:

[0011] A data processing module, configured to obtain a training data set and perform feature extraction processing on the sample data in the training data set;

[0012] A model training module, configured to input the processed sample data and the corresponding sentiment label of the sample data into an attention recurrent convolutional network for model training to obtain a sentiment recognition model;

[0013] Wherein, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism;

[0014] A sentiment recognition module, configured to input the target electroencephalogram signal to be recognized into the sentiment recognition model, and use the output result of the sentiment recognition model as the sentiment recognition result of the target electroencephalogram signal.

[0015] According to another aspect of the present invention, there is provided an electronic device, including:

[0016] At least one processor; and

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the sentiment recognition method according to any embodiment of the present invention.

[0019] According to another aspect of the present invention, there is provided a computer-readable storage medium storing computer instructions for causing a processor to implement the sentiment recognition method according to any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention solves the problem of low efficiency and accuracy of sentiment recognition of time series networks such as RNN by obtaining a training data set, performing feature extraction processing on the sample data in the training data set, inputting the processed sample data and the corresponding sentiment label of the sample data into an attention recurrent convolutional network for model training to obtain a sentiment recognition model, wherein the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism, inputting the target electroencephalogram signal to be recognized into the sentiment recognition model, and using the output result of the sentiment recognition model as the sentiment recognition result of the target electroencephalogram signal, achieving the beneficial effect of constructing a new sentiment recognition model and improving the efficiency and accuracy of sentiment recognition based on electroencephalogram signals.

[0021] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0023] Figure 1 is a flowchart of an emotion recognition method provided in Embodiment 1 of the present invention;

[0024] Figure 2 is a flowchart of an emotion recognition method provided in Embodiment 2 of the present invention;

[0025] Figure 3 is an overall block diagram of the emotion recognition method applicable to Embodiment 2 of the present invention;

[0026] Figure 4 is a schematic structural diagram of a pre-attention recurrent convolutional network applicable to Embodiment 2 of the present invention;

[0027] Figure 5 is a schematic structural diagram of a post-attention recurrent convolutional network applicable to Embodiment 2 of the present invention;

[0028] Figure 6 is a schematic structural diagram of an emotion recognition device provided in Embodiment 3 of the present invention;

[0029] Figure 7 is a schematic structural diagram of an electronic device for implementing the emotion recognition method of the embodiments of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0030] In order to enable those skilled in the art to better understand the solutions of the present invention, the following clearly and completely describes the technical solutions in the embodiments of the present invention with reference to the drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0031] It should be noted that the term "target EEG signal" in the description, claims and above-mentioned drawings of the present invention is used to distinguish similar objects, and does not necessarily describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present invention described here can be implemented in an order different from those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0032] Embodiment 1

[0033] Figure 1 is a flowchart of an emotion recognition method provided according to Embodiment 1 of the present invention. This embodiment is applicable to the situation of quickly and accurately realizing emotion classification based on EEG signals. This method can be executed by an emotion recognition device, which can be implemented in the form of hardware and / or software, and the emotion recognition device can be configured in an electronic device. As Figure 1 shown, the method includes:

[0034] S110. Obtain a training data set and perform feature extraction processing on the sample data in the training data set.

[0035] In this embodiment, the training data set includes multiple EEG signals for emotion classification training of a preset attention recurrent convolutional network model. Optionally, the training data set is the DEAP data set, which records the EEG signals of 32 testers. That is, 32 testers are respectively asked to watch 40 one-minute-long different types of music video clips, and the lead EEG signals are collected. Then, the testers respectively score the video music they see according to different criteria, and the final data is in the format of 40×32×8064. Among them, 40 represents the 40 video clips watched by the testers, 32 represents the number of leads used in the experiment, and 8064 represents sampling the testers at a frequency of 128 Hz during a 63s segment to obtain 8064 data points.

[0036] Optionally, performing feature extraction processing on the sample data in the training data set includes: performing downsampling processing and filtering processing on the sample data in the training data set; performing time-frequency domain transformation on the processed sample data using the wavelet packet decomposition algorithm; calculating the corresponding wavelet packet coefficient values according to the transformation results as the feature values of the sample data.

[0037] In this embodiment, after obtaining the training data set, the EEG signals in the training data set are first preprocessed, that is, downsampling and filtering operations are performed. Then, the wavelet packet decomposition algorithm is used to decompose the low-frequency part and the high-frequency part of the EEG signals, perform time-frequency domain transformation, and finally calculate the corresponding wavelet packet coefficient values as the eigenvalue of the extracted EEG signals.

[0038] S120. Input the processed sample data and the corresponding emotion labels into the attention recurrent convolutional network for model training to obtain an emotion recognition model.

[0039] In this embodiment, while extracting features from the sample data, the constructed emotion classification model can be used to label the sample data so that each sample data has a corresponding emotion label. The sample data after feature extraction and the corresponding emotion labels are input into a pre-set attention recurrent convolutional network for model training, and the trained model is used as an emotion recognition model to select the optimal emotion recognition result based on EEG signals.

[0040] Among them, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism. As Figure 2 shown, the one-dimensional convolutional neural network and the bidirectional simple recurrent unit form a recurrent convolutional neural network, and combined with the attention mechanism, an attention recurrent convolutional network is generated.

[0041] Among them, the one-dimensional convolutional neural network is a special form of CNN, which can process ordered EEG signals and mainly includes a convolutional layer, a pooling layer, and a fully connected layer. Its input is a vector and a convolution kernel, and the output is a vector. Its essence is a multi-layer perceptron. Usually, the length of the input vector is greater than the length of the convolution kernel, and the length of the output vector depends on the padding scheme of the convolution operation. The simple recurrent unit SRU belongs to a special form of RNN, which can improve the disadvantages of the traditional RNN with long calculation time and long-term dependence, so that while maintaining the original characteristics of the RNN, the calculation speed is improved, and the two reach a balance.

[0042] The attention mechanism uses the limited attention resources of humans to focus on more important information to improve the efficiency and accuracy of obtaining information. Since EEG signals are a kind of sequence signals and there is a certain correlation between the front and the back, once the waveform changes at different moments, it can be considered that the individual's emotion has changed. With the help of the attention mechanism, these changes can be better discovered and remembered, and further analysis can be performed based on these features, so that the model can better judge emotions.

[0043] S130. Input the target EEG signal to be recognized into the emotion recognition model, and use the output result of the emotion recognition model as the emotion recognition result of the target EEG signal.

[0044] In this embodiment, the target EEG signal can be any EEG signal to be subjected to emotion classification. Optionally, the target EEG signal to be recognized is input into the emotion recognition model, and the output result of the emotion recognition model is used as the emotion recognition result of the target EEG signal, including: performing downsampling processing and filtering processing on the target EEG signal, and performing feature extraction processing on the filtered target EEG signal by using the wavelet packet decomposition algorithm; inputting the target EEG signal after feature extraction into the emotion recognition model, and obtaining the output result of the emotion recognition model as the emotion recognition result of the target EEG signal.

[0045] In the embodiment of the present invention, feature extraction processing is performed on the sample data in the training data set; the processed sample data and the emotion labels are input into the attention recurrent convolutional network for model training to obtain an emotion recognition model; wherein, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism; the target EEG signal to be recognized is input into the emotion recognition model, and the output result of the emotion recognition model is used as the emotion recognition result of the target EEG signal, which solves the problem that the efficiency and accuracy of emotion recognition of time series networks such as RNN are relatively low, and achieves the beneficial effects of constructing a new emotion recognition model and improving the efficiency and accuracy of emotion recognition based on EEG signals.

[0046] Embodiment 2

[0047] Figure 2 is a flowchart of an emotion recognition method provided according to Embodiment 2 of the present invention. On the basis of the above embodiment, this embodiment further provides the specific steps of inputting the processed sample data and the emotion labels corresponding to the sample data into the attention recurrent convolutional network for model training to obtain an emotion recognition model. As Figure 2 shown, the method includes:

[0048] S210. Obtain a training data set, and perform feature extraction processing on the sample data in the training data set.

[0049] S220. Input the processed sample data and the emotion labels corresponding to the sample data into the attention recurrent convolutional network for model training to obtain an emotion recognition model; the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism.

[0050] Optionally, the attention recurrent convolutional network includes: a soft attention recurrent convolutional network and a self-attention recurrent convolutional network; the soft attention recurrent convolutional network includes: a pre-soft attention recurrent convolutional network and a post-soft attention recurrent convolutional network; the self-attention recurrent convolutional network includes: a pre-self-attention recurrent convolutional network and a post-self-attention recurrent convolutional network.

[0051] In this embodiment, as Figure 3 shown, if the recurrent convolutional neural network is combined with the soft attention mechanism, a soft attention recurrent convolutional network is generated; if the recurrent convolutional neural network is combined with the self-attention mechanism, a self-attention recurrent convolutional network is generated. Each attention recurrent convolutional network can be configured into two different frameworks, namely, the front-end framework and the back-end framework. The main difference between the two is the position where the attention layer processes data in the attention recurrent convolutional network.

[0052] Optionally, the processed sample data and the corresponding sentiment labels of the sample data are input into the attention recurrent convolutional network for model training to obtain a sentiment recognition model, including: if the attention recurrent convolutional network is a front-end soft attention recurrent convolutional network or a front-end self-attention recurrent convolutional network, the attention layer in the attention recurrent convolutional network assigns corresponding weights to the input sample data to obtain corrected sample data; the recurrent convolutional network layer performs feature extraction operations on the corrected sample data, and combines with the corresponding sentiment labels of the sample data for model training to obtain a sentiment recognition model.

[0053] In this embodiment, as Figure 4 shown, when the attention recurrent convolutional network uses the front-end framework, the input sample data x i first enters the attention layer to assign corresponding weights to obtain corrected sample data Then it is input into the recurrent convolutional network layer for feature extraction operations, and combines with the corresponding sentiment labels of the sample data for model training, and finally obtains a trained sentiment recognition model.

[0054] Optionally, the processed sample data and the corresponding sentiment labels of the sample data are input into the attention recurrent convolutional network for model training to obtain a sentiment recognition model, including: if the attention recurrent convolutional network is a back-end soft attention recurrent convolutional network or a back-end self-attention recurrent convolutional network, the recurrent convolutional network layer in the attention recurrent convolutional network performs feature extraction operations on the input sample data; the attention layer assigns corresponding weights to the sample data after feature extraction operations, and combines with the corresponding sentiment labels of the sample data for model training to obtain a sentiment recognition model.

[0055] In this embodiment, as Figure 5 shown, when the attention recurrent convolutional network uses the back-end framework, the input sample data x i first enters the recurrent convolutional network layer for feature extraction operations, and then an attention layer is added before sentiment classification of the sample data to assign corresponding weights to obtain corrected sample data Finally Perform further sentiment classification and combine it with the sentiment labels corresponding to the data for model training, and finally obtain a trained sentiment recognition model.

[0056] In this embodiment, through Figure 4 and Figure 5 The difference between the front frame and the rear frame can be shown more directly. The front attention layer processes the original EEG signals that have not been processed yet. At this moment, there is no correlation or a relatively low correlation between the signal sequence segments; while the rear attention layer analyzes the EEG signals after feature extraction. The signal sequences have interacted with each other, and their mutual correlation is higher, so the accuracy of sentiment classification is higher.

[0057] S230. Input the target EEG signal to be recognized into the sentiment recognition model, and use the output result of the sentiment recognition model as the sentiment recognition result of the target EEG signal.

[0058] In this embodiment, the sentiment recognition model combines a one-dimensional convolutional neural network and a bidirectional simple recurrent unit, making the classification effect of the model better than that of using only CNN or only SRU. It can be applied to EEG signals with temporal characteristics, solving the problem of the large computational complexity of temporal networks such as RNN, and having both the sequence relationship of RNN and the rapid calculation of CNN. And the attention mechanism is adopted in the model, which can increase the front-back correlation in the input data and correspondingly improve the accuracy of sentiment recognition.

[0059] Embodiment III

[0060] Figure 6 is a schematic structural diagram of a sentiment recognition device provided according to Embodiment III of the present invention.

[0061] As Figure 6 shown, the device includes:

[0062] A data processing module 610, configured to obtain a training data set and perform feature extraction processing on the sample data in the training data set;

[0063] A model training module 620, configured to input the processed sample data and the sentiment labels corresponding to the sample data into an attention recurrent convolutional network for model training to obtain a sentiment recognition model;

[0064] wherein, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism;

[0065] A sentiment recognition module 630, configured to input the target EEG signal to be recognized into the sentiment recognition model, and use the output result of the sentiment recognition model as the sentiment recognition result of the target EEG signal.

[0066] Optionally, the data processing module 610 is configured to:

[0067] Perform downsampling processing and filtering processing on the sample data in the training data set;

[0068] Perform time-frequency domain transformation on the processed sample data by using the wavelet packet decomposition algorithm;

[0069] Calculate the corresponding wavelet packet coefficient value according to the transformation result as the eigenvalue of the sample data.

[0070] Optionally, the training data set is the DEAP data set.

[0071] Optionally, the attention recurrent convolutional network includes: a soft attention recurrent convolutional network and a self-attention recurrent convolutional network;

[0072] The soft attention recurrent convolutional network includes: a pre-soft attention recurrent convolutional network and a post-soft attention recurrent convolutional network;

[0073] The self-attention recurrent convolutional network includes: a pre-self-attention recurrent convolutional network and a post-self-attention recurrent convolutional network.

[0074] Optionally, the model training module 620 includes:

[0075] A first training unit, configured to, if the attention recurrent convolutional network is a pre-soft attention recurrent convolutional network or a pre-self-attention recurrent convolutional network, allocate corresponding weights to the input sample data through the attention layer in the attention recurrent convolutional network to obtain corrected sample data;

[0076] Perform feature extraction operations on the corrected sample data through the recurrent convolutional network layer, and perform model training in combination with the sentiment label corresponding to the sample data to obtain a sentiment recognition model.

[0077] Optionally, the model training module 620 includes:

[0078] A second training unit, configured to, if the attention recurrent convolutional network is a post-soft attention recurrent convolutional network or a post-self-attention recurrent convolutional network, perform feature extraction operations on the input sample data through the recurrent convolutional network layer in the attention recurrent convolutional network;

[0079] Allocate corresponding weights to the sample data after the feature extraction operation through the attention layer, and perform model training in combination with the sentiment label corresponding to the sample data to obtain a sentiment recognition model.

[0080] Optionally, the sentiment recognition module 630 is configured to:

[0081] Downsample and filter the target EEG signal, and use the wavelet packet decomposition algorithm to extract features from the filtered target EEG signal;

[0082] Input the target EEG signal after feature extraction into the emotion recognition model, and obtain the output result of the emotion recognition model as the emotion recognition result of the target EEG signal.

[0083] The emotion recognition device provided by the embodiments of the present invention can execute the emotion recognition method provided by any embodiment of the present invention, and has corresponding functional modules and beneficial effects for executing the method.

[0084] Embodiment Four

[0085] Figure 7 FIG. shows a schematic structural diagram of an electronic device 10 that can be used to implement the embodiments of the present invention. The electronic device is intended to represent various forms of digital computers, such as, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0086] As Figure 7 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can execute various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0087] A plurality of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0088] The processor 11 may be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the emotion recognition method.

[0089] In some embodiments, the emotion recognition method may be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the emotion recognition method described above may be executed. Alternatively, in other embodiments, the processor 11 may be configured to execute the emotion recognition method by any other suitable means (e.g., by means of firmware).

[0090] The various embodiments of the systems and techniques described above in this document may be implemented in digital electronic circuitry, integrated circuit systems, field-programmable gate arrays (FPGA), application-specific integrated circuits (ASIC), application-specific standard products (ASSP), systems-on-chip (SOC), complex programmable logic devices (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a special-purpose or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0091] The computer program for implementing the method of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowchart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0092] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0093] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0094] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0095] A computing system may include a client and a server. The client and the server are generally far from each other and usually interact via a communication network. The client-server relationship is created by computer programs running on respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or a cloud host, which is a host product in the cloud computing service system, addressing the defects of high management difficulty and weak business scalability existing in traditional physical hosts and VPS services.

[0096] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is imposed herein.

[0097] The above specific embodiments do not constitute a limitation to the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

Claims

1. An emotion recognition method, characterized in that Including: Obtain a training data set, and perform feature extraction processing on the sample data in the training data set; Input the processed sample data and the corresponding sentiment labels of the sample data into an attention recurrent convolutional network for model training to obtain a sentiment recognition model; Among them, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism; Input the target electroencephalogram signal to be recognized into the sentiment recognition model, and use the output result of the sentiment recognition model as the sentiment recognition result of the target electroencephalogram signal; Among them, the attention recurrent convolutional network includes: a soft attention recurrent convolutional network or a self-attention recurrent convolutional network; The soft attention recurrent convolutional network includes: a pre-soft attention recurrent convolutional network or a post-soft attention recurrent convolutional network; The self-attention recurrent convolutional network includes: a pre-self-attention recurrent convolutional network or a post-self-attention recurrent convolutional network; Inputting the processed sample data and the corresponding sentiment labels of the sample data into an attention recurrent convolutional network for model training to obtain a sentiment recognition model includes: If the attention recurrent convolutional network is a pre-soft attention recurrent convolutional network or a pre-self-attention recurrent convolutional network, the attention layer in the attention recurrent convolutional network assigns corresponding weights to the input sample data to obtain corrected sample data, and the recurrent convolutional network layer performs feature extraction operations on the corrected sample data, and combines with the corresponding sentiment labels of the sample data for model training to obtain a sentiment recognition model; If the attention recurrent convolutional network is a post-soft attention recurrent convolutional network or a post-self-attention recurrent convolutional network, the recurrent convolutional network layer in the attention recurrent convolutional network performs feature extraction operations on the input sample data, and the attention layer assigns corresponding weights to the sample data after the feature extraction operation to obtain corrected sample data, and the corrected sample data is combined with the corresponding sentiment labels of the sample data for model training to obtain a sentiment recognition model.

2. The method according to claim 1, wherein Performing feature extraction processing on the sample data in the training data set includes: Performing downsampling processing and filtering processing on the sample data in the training data set; Performing time-frequency domain transformation on the processed sample data using the wavelet packet decomposition algorithm; Calculating the corresponding wavelet packet coefficient values according to the transformation results as the feature values of the sample data.

3. The method according to claim 1, characterized in that, The training data set is the DEAP data set.

4. The method according to claim 1, wherein Inputting the target electroencephalogram signal to be recognized into the sentiment recognition model, and using the output result of the sentiment recognition model as the sentiment recognition result of the target electroencephalogram signal includes: Performing downsampling processing and filtering processing on the target electroencephalogram signal, and performing feature extraction processing on the filtered target electroencephalogram signal using the wavelet packet decomposition algorithm; Inputting the target electroencephalogram signal after feature extraction into the sentiment recognition model, and obtaining the output result of the sentiment recognition model as the sentiment recognition result of the target electroencephalogram signal.

5. An emotion recognition device, characterized in that, Including: A data processing module for obtaining a training data set and performing feature extraction processing on the sample data in the training data set; A model training module, configured to perform model training by inputting the processed sample data and the corresponding sentiment label of the sample data into an attention recurrent convolutional network to obtain a sentiment recognition model; Wherein, the attention recurrent convolutional network includes a one-dimensional convolutional neural network, a bidirectional simple recurrent unit, and an attention mechanism; A sentiment recognition module, configured to perform inputting the target EEG signal to be recognized into the sentiment recognition model, and taking the output result of the sentiment recognition model as the sentiment recognition result of the target EEG signal; Wherein, the attention recurrent convolutional network includes: a soft attention recurrent convolutional network or a self-attention recurrent convolutional network; The soft attention recurrent convolutional network includes: a pre-soft attention recurrent convolutional network or a post-soft attention recurrent convolutional network; The self-attention recurrent convolutional network includes: a pre-self-attention recurrent convolutional network or a post-self-attention recurrent convolutional network; The model training module includes: A first training unit, configured to, if the attention recurrent convolutional network is a pre-soft attention recurrent convolutional network or a pre-self-attention recurrent convolutional network, assign corresponding weights to the input sample data through the attention layer in the attention recurrent convolutional network to obtain corrected sample data, perform feature extraction operations on the corrected sample data through the recurrent convolutional network layer, and perform model training in combination with the sentiment label corresponding to the sample data to obtain a sentiment recognition model; A second training unit, configured to, if the attention recurrent convolutional network is a post-soft attention recurrent convolutional network or a post-self-attention recurrent convolutional network, perform feature extraction operations on the input sample data through the recurrent convolutional network layer in the attention recurrent convolutional network, assign corresponding weights to the sample data after the feature extraction operation through the attention layer to obtain corrected sample data, and perform model training on the corrected sample data and the sentiment label corresponding to the sample data to obtain a sentiment recognition model.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the sentiment recognition method according to any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, and the computer instructions are used to implement the sentiment recognition method according to any one of claims 1-4 when executed by a processor.

Images