Emotion recognition method, device and electronic device based on brain waves

Abstract

The embodiment of the present application provides a method, device and electronic device for emotion recognition based on brain waves, and relates to the field of computer technology. In the embodiment of the present application, after obtaining the power spectrum density of the brain wave signal of the target object, the frequency band energy corresponding to the α wave, theta wave and the beta wave that can reflect the target object's emotional peace state can be determined, and the frequency band energy corresponding to the first harmonic and the second harmonic generated by the Schumann wave resonance that can affect the target object's emotions can be determined. In this way, the relaxation degree obtained by the frequency band energy corresponding to the α wave, theta wave and the beta wave, and the Schumann resonance energy index obtained based on the frequency band range corresponding to the α wave, theta wave and the beta wave, and the frequency band energy corresponding to the first harmonic and the second harmonic of the Schumann resonance, can be used to perform emotion recognition on the target object, and the emotion recognition result of the target object can be obtained more accurately.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device and electronic device for emotion recognition based on brain waves. Background Art

[0002] Brain waves are electrical signals generated by the activity of brain neurons, and can be divided into delta waves (frequency range: 0.5-4 Hz), theta waves (frequency range: 4-8 Hz), alpha waves (frequency range: 8-13Hz), beta waves (frequency range: 13-30 Hz) and gamma waves (frequency range: >30 Hz) according to the frequency of the electrical signals. Each brain wave frequency corresponds to a specific psychological state or consciousness activity, among which alpha waves correspond to the target object being in a peaceful state, that is, a high degree of relaxation.

[0003] Therefore, by identifying the strength of the α wave in the EEG signal, it is possible to determine whether the target object is in a peaceful state. For example, if the α wave in the EEG signal is strong, it can be determined that the target object is in a peaceful state; if the α wave in the EEG signal is weak, it can be determined that the target object is not in a peaceful state.

[0004] However, using the above-mentioned EEG-based emotion recognition method, judging whether the target object is in a peaceful state only based on the strength of the alpha wave in the EEG signal may lead to inaccurate emotion recognition results, that is, the accuracy of emotion recognition is low. Summary of the invention

[0005] The embodiments of the present application provide a method, device and electronic device for emotion recognition based on brain waves, which are used to accurately determine the peaceful state of a target object, thereby improving the accuracy of emotion recognition.

[0006] In a first aspect, an embodiment of the present application provides a method for emotion recognition based on brain waves, the method comprising:

[0007] The power spectrum density of the EEG signal of the target object is estimated to obtain the power spectrum density of the EEG signal; the power spectrum density represents the power distribution of the EEG signal in the frequency domain;

[0008] The relaxation degree of the target object is obtained based on the energy of the frequency bands corresponding to the alpha wave, theta wave and beta wave in the EEG signal determined by the power spectrum density;

[0009] Based on the frequency bands corresponding to the alpha wave, the theta wave and the beta wave, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann wave resonance, the Schumann resonance energy index corresponding to the first harmonic and the second harmonic is obtained; the Schumann resonance energy index represents the degree of influence of the Schumann wave on the brain of the target object;

[0010] Based on the relaxation degree and Schumann resonance energy index, the emotion recognition result corresponding to the EEG signal is determined.

[0011] In an optional embodiment, before estimating the power spectrum density of the EEG signal of the target object and obtaining the power spectrum density, the method further includes:

[0012] Acquire the original signals of the target object collected in the preset multiple EEG signal collection areas, and pre-process the original signals to obtain the pre-processed original signals;

[0013] The preprocessed original signal is subjected to artifact removal and re-reference processing in sequence to obtain the EEG signal.

[0014] In an optional embodiment, artifact removal and re-reference processing are sequentially performed on the pre-processed original signal to obtain an EEG signal, including:

[0015] The preprocessed original signal is subjected to blind source separation and artifact removal processing in sequence to obtain the original signal after artifact removal;

[0016] The original signal after artifact removal is re-referenced to obtain the EEG signal.

[0017] In an optional embodiment, based on the frequency band ranges corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance, the Schumann resonance energy index corresponding to the first harmonic and the second harmonic is obtained, including:

[0018] Determine frequency band energies corresponding to the first harmonic and the second harmonic respectively based on the frequency band ranges set for the first harmonic and the second harmonic respectively;

[0019] Based on the frequency band ranges corresponding to the first harmonic and the second harmonic, and the frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively, determining the sub-frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively;

[0020] Based on the sub-frequency band ranges and power spectrum densities corresponding to the α wave, the θ wave and the β wave, respectively, the sub-frequency band energies corresponding to the α wave, the θ wave and the β wave, respectively, are determined;

[0021] The Schumann resonance energy index is obtained based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic.

[0022] In an optional embodiment, the sum of the sub-frequency band range corresponding to the α wave and the sub-frequency band range corresponding to the θ wave includes the frequency band range corresponding to the first harmonic, and the sub-frequency band range corresponding to the β wave includes the frequency band range corresponding to the second harmonic.

[0023] In an optional embodiment, based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic, the Schumann resonance energy index is obtained, including:

[0024] The ratio of the frequency band energy corresponding to the first harmonic to the sum of the energy of the sub-frequency band corresponding to the α wave and the energy of the sub-frequency band corresponding to the θ wave is taken as the first energy ratio;

[0025] The ratio of the frequency band energy corresponding to the second harmonic to the sub-frequency band energy corresponding to the β wave is used as the second energy ratio;

[0026] A Schumann resonance energy index is obtained based on the first energy ratio and the second energy ratio.

[0027] In an optional embodiment, the method further includes:

[0028] Determine the coefficient of variation of the frequency band energy corresponding to alpha waves, theta waves and beta waves respectively;

[0029] Based on the relaxation degree and the Schumann resonance energy index, the emotion recognition result corresponding to the EEG signal is determined, further including:

[0030] The emotion recognition results are determined based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α waves, θ waves and β waves respectively.

[0031] In an optional embodiment, the emotion recognition result is determined based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to the alpha wave, the theta wave and the beta wave respectively, including:

[0032] The coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α wave, θ wave and β wave are input into the pre-trained emotion recognition model to obtain the prediction probabilities corresponding to various recognition results.

[0033] The emotion recognition result is determined based on the prediction probabilities corresponding to the various recognition results.

[0034] In a second aspect, the present application also provides an emotion recognition device based on brain waves, the device comprising:

[0035] A signal estimation module is used to estimate the power spectrum density of the EEG signal of the target object to obtain the power spectrum density of the EEG signal; the power spectrum density represents the power distribution of the EEG signal in the frequency domain;

[0036] The first processing module is used to obtain the relaxation degree of the target object based on the frequency band energy corresponding to the alpha wave, theta wave and beta wave in the EEG signal determined by the power spectrum density;

[0037] The second processing module is used to obtain the Schumann resonance energy index corresponding to the first harmonic and the second harmonic based on the frequency band ranges corresponding to the alpha wave, the theta wave and the beta wave respectively, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance respectively; the Schumann resonance energy index represents the degree of influence of the Schumann wave on the brain of the target object;

[0038] The emotion recognition module is used to determine the emotion recognition result corresponding to the EEG signal based on the relaxation degree and the Schumann resonance energy index.

[0039] In an optional embodiment, before performing power spectrum density estimation on the EEG signal of the target object to obtain the power spectrum density, the signal estimation module is further used to:

[0040] Acquire the original signals of the target object collected in the preset multiple EEG signal collection areas, and pre-process the original signals to obtain the pre-processed original signals;

[0041] The preprocessed original signal is subjected to artifact removal and re-reference processing in sequence to obtain the EEG signal.

[0042] In an optional embodiment, when performing artifact removal and re-reference processing on the pre-processed original signal in sequence to obtain the EEG signal, the signal estimation module is specifically used to:

[0043] The preprocessed original signal is subjected to blind source separation and artifact removal processing in sequence to obtain the original signal after artifact removal;

[0044] The original signal after artifact removal is re-referenced to obtain the EEG signal.

[0045] In an optional embodiment, when obtaining the Schumann resonance energy index corresponding to the first harmonic and the second harmonic based on the frequency band ranges corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance, the first processing module is specifically used to:

[0046] Determine frequency band energies corresponding to the first harmonic and the second harmonic respectively based on the frequency band ranges set for the first harmonic and the second harmonic respectively;

[0047] Based on the frequency band ranges corresponding to the first harmonic and the second harmonic, and the frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively, determining the sub-frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively;

[0048] Based on the sub-frequency band ranges and power spectrum densities corresponding to the α wave, the θ wave and the β wave, respectively, the sub-frequency band energies corresponding to the α wave, the θ wave and the β wave, respectively, are determined;

[0049] The Schumann resonance energy index is obtained based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic.

[0050] In an optional embodiment, when obtaining the Schumann resonance energy index based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic, the second processing module is specifically used to:

[0051] The ratio of the frequency band energy corresponding to the first harmonic to the sum of the energy of the sub-frequency band corresponding to the α wave and the energy of the sub-frequency band corresponding to the θ wave is taken as the first energy ratio;

[0052] The ratio of the frequency band energy corresponding to the second harmonic to the sub-frequency band energy corresponding to the β wave is used as the second energy ratio;

[0053] A Schumann resonance energy index is obtained based on the first energy ratio and the second energy ratio.

[0054] In an optional embodiment, the second processing module is further used for:

[0055] Determine the coefficient of variation of the frequency band energy corresponding to alpha waves, theta waves and beta waves respectively;

[0056] When determining the emotion recognition result corresponding to the EEG signal based on the relaxation degree and the Schumann resonance energy index, the emotion recognition module is specifically used to:

[0057] The emotion recognition results are determined based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α waves, θ waves and β waves respectively.

[0058] In an optional embodiment, when determining the emotion recognition result based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to the alpha wave, the theta wave and the beta wave respectively, the emotion recognition module is specifically used to:

[0059] The coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α wave, θ wave and β wave are input into the pre-trained emotion recognition model to obtain the prediction probabilities corresponding to various recognition results.

[0060] The emotion recognition result is determined based on the prediction probabilities corresponding to the various recognition results.

[0061] In a third aspect, an embodiment of the present application further provides an electronic device, including:

[0062] Processor; and

[0063] Memory for storing programs,

[0064] The program includes instructions, which, when executed by a processor, cause the processor to execute the brain wave-based emotion recognition method as described in the first aspect.

[0065] In a fourth aspect, an embodiment of the present application further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the brain wave-based emotion recognition method as described in the first aspect.

[0066] In a fifth aspect, the present application provides a computer program product, which, when called by a computer, enables the computer to execute the steps of the brain wave-based emotion recognition method as described in the first aspect.

[0067] The beneficial effects of this application are as follows:

[0068] In the brain wave-based emotion recognition method provided in the embodiment of the present application, after the power spectrum density of the brain wave signal of the target object is estimated and the power spectrum density of the brain wave signal is obtained, the frequency band energy corresponding to the α wave, the θ wave and the β wave that can reflect the target object's emotional peace state can be determined, and the frequency band energy corresponding to the first harmonic and the second harmonic generated by the Schumann wave resonance that can affect the target object's emotions can be determined. In this way, the relaxation degree of the target object obtained by the frequency band energy corresponding to the α wave, the θ wave and the β wave, and the Schumann resonance energy index corresponding to the first harmonic and the second harmonic obtained based on the frequency band range corresponding to the α wave, the θ wave and the β wave, and the frequency band energy corresponding to the first harmonic and the second harmonic of the Schumann resonance, can be used to accurately obtain the target object's emotion recognition result (for example, peaceful state), thereby improving the accuracy of emotion recognition.

[0069] In addition, other features and advantages of the present application will be described in the subsequent description, and partly become apparent from the description, or be understood by practicing the present application. The purpose and other advantages of the present application can be realized and obtained by the structures particularly pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described here are used to provide a further understanding of the present application, constitute a part of the present application, and do not constitute an improper limitation on the present application. In the drawings:

[0071] Figure 1 A schematic diagram of an optional system architecture applicable to the embodiments of the present application;

[0072] Figure 2A schematic diagram of an implementation flow of a method for emotion recognition based on brain waves provided in an embodiment of the present application;

[0073] Figure 3 A schematic diagram of the specific distribution of an EEG signal collection area provided in an embodiment of the present application;

[0074] Figure 4 A schematic diagram of an implementation flow of a method for obtaining a Schumann resonance energy index provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of a specific application scenario of emotion recognition provided in an embodiment of the present application;

[0076] Figure 6 A logical schematic diagram of emotion recognition provided in an embodiment of the present application;

[0077] Figure 7 A specific schematic diagram of the correlation between the characteristics of a model evaluation and the peacefulness of a target object provided in an embodiment of the present application;

[0078] Figure 8 A method based on the embodiment of the present application is provided Figure 2 Schematic diagram of the

[0079] Fig. 9 A schematic diagram of the structure of an emotion recognition device based on brain waves provided in an embodiment of the present application;

[0080] Fig.10 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0081] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0082] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0083] The term "including" and its variations used in this document are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions of other terms will be given in the description below. It should be noted that the concepts of "first", "second", etc. mentioned in this application are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0084] It should be noted that the modifications of "one" and "plurality" mentioned in the present application are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0085] The names of the messages or information exchanged between multiple devices in the embodiments of the present application are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0086] Some of the terms used in the embodiments of the present application are explained below to facilitate understanding by those skilled in the art.

[0087] (1) Schumann resonance (SR): It is an extremely low frequency electromagnetic wave generated by the resonance between the earth and the ionosphere. The main frequency is around 7.83 Hz, and its harmonic frequencies are 14.3 Hz, 20.8 Hz, etc.

[0088] (2) Level of peace: also known as brain level peace, is a comprehensive indicator that measures an individual's brain relaxation, inner peace, and emotional stability within a specific period of time.

[0089] (3) Power spectral density (PSD): It is an indicator used to describe the distribution of signal power in the frequency domain. It can express the relationship between the signal power and frequency within a unit frequency band. Power spectral density is usually measured in watts per hertz (W / Hz). By analyzing the power spectral density, we can understand the frequency ranges in which the signal power is mainly concentrated, and the relative size of the power at different frequencies.

[0090] (4) Welch method: It is a power spectral density estimation method used to analyze time series data. Since the signal is segmented and the power spectral density estimation value of each segment is averaged to reduce the impact of noise, it is suitable for situations where the signal contains noise.

[0091] (5) Independent components analysis (ICA): It is a signal processing technique used to separate independent source signals from a mixed signal of multiple source signals.

[0092] (6) The Fast ICA algorithm is a fast iterative algorithm for ICA. It uses batch processing to process a large amount of sample data in each iteration, allowing the algorithm to efficiently process high-dimensional data.

[0093] (7) EEG: Also known as electroencephalogram (EEG): It is a bioelectric signal that reflects the activity state of the brain and contains rich physiological and pathological information. EEG signals can be obtained by measuring the voltage changes generated by brain cell groups. EEG signals can be divided into spontaneous EEG and evoked EEG. Among them, spontaneous EEG can be divided into different bands such as α wave, θ wave and β wave according to different frequencies.

[0094] (8) Blind source separation: Also known as blind source separation (BSS), it refers to the process of separating the source signals from the aliased signal (observation signal) when the theoretical model of the signal and the source signal cannot be accurately known.

[0095] (9) Artifacts: refers to various forms of images that appear on the image even though the scanned object does not exist.

[0096] (10) Re-referencing: It is an important step in EEG signal processing. Its purpose is to eliminate the influence of the reference electrode on the original signal, thereby improving the signal quality and the accuracy of subsequent analysis.

[0097] (11) Leave-one-out (LOO) is a model validation technique. Its core idea is to divide the data set into k equal-sized subsets (or "folds"); select only one sample from each of the k subsets as the test set; use the remaining k-1 samples as the training set; repeat the above steps k times, selecting a different sample as the test set each time; and average the results of each test to obtain an overall estimate of the model performance. In this way, each sample is used as a test set, so a more accurate estimate of the model performance can be obtained.

[0098] (12) Coefficient of variation (CV): It is a statistic used to measure the degree of variation in data and can be determined by the ratio of the standard deviation (SD) to the mean.

[0099] Based on the above-mentioned nouns and related terminology explanations, the design concept of the embodiments of the present application is briefly introduced below:

[0100] Brain waves are electrical signals generated by the activity of brain neurons. Brain waves can be divided into different types according to the frequency of the signal. Each brain wave frequency corresponds to a specific psychological state or conscious activity. The brain will be dominated by brain waves of different frequencies in different states. For example, brain waves can be divided into five main frequency bands according to the size of the frequency, and each frequency band is related to a specific state of consciousness and psychological activity.

[0101] Among them, delta waves have the lowest frequency and are usually associated with deep sleep and unconsciousness. Delta waves play a key role in body repair, immune system enhancement, and tissue regeneration. During deep, dreamless slow-wave sleep, delta wave activity dominates, helping the body recover physically. Theta waves have a slightly higher frequency than delta waves, usually appearing in light sleep, deep relaxation, and meditation, and are also associated with creative thinking and intuition. Theta waves reflect the state of the brain entering the subconscious, which helps with emotional processing and inspiration. Therefore, theta wave activity is significantly enhanced during meditation, fantasy, and deep relaxation. Alpha waves have a slightly higher frequency than theta waves, representing a state of wakefulness and relaxation, usually occurring when closing your eyes, meditating, or sitting quietly to rest. Alpha waves help coordinate information processing in various areas of the brain, improve integration capabilities, and put the brain in a comfortable and non-tense state. In this state, people tend to feel calm and focused. Beta waves have a higher frequency than alpha waves, representing a state of active, focused, and logical thinking in the brain. Beta wave activity increases significantly when we study, work, solve problems, or perform other complex cognitive activities. However, excessive beta wave activity can be associated with states such as anxiety, tension and stress, and excessive mental concentration can trigger negative emotions. Gamma waves are the highest frequency brain waves and are generally associated with higher cognitive functions, including information processing, memory consolidation and conscious awareness. Gamma wave activity increases during periods of high concentration, complex thinking or creative activities. Gamma waves play an important role in effective communication between different areas of the brain, helping to improve perceptual clarity and the ability to process complex information.

[0102] Therefore, by identifying the strength of the α wave in the EEG signal, it is possible to determine whether the target object is in a peaceful state. For example, if the α wave in the EEG signal is strong, it can be determined that the target object is in a peaceful state; if the α wave in the EEG signal is weak, it can be determined that the target object is not in a peaceful state.

[0103] However, using the above-mentioned EEG-based emotion recognition method, judging whether the target object is in a peaceful state only based on the strength of the alpha wave in the EEG signal may lead to inaccurate emotion recognition results, that is, the accuracy of emotion recognition is low.

[0104] There is a significant correlation between the Schumann resonance and the frequency band range corresponding to the α wave, which helps the brain enter a state of relaxation and meditation. Therefore, based on the Schumann resonance and the frequency analysis of brain waves, a new emotion recognition method can be provided to identify and quantify the peaceful state of the target object, so as to more accurately judge the peacefulness of the target object, thereby improving the accuracy of emotion recognition. In an optional implementation, the embodiment of the present application proposes an emotion recognition method based on brain waves, which can specifically include: estimating the power spectrum density of the brain wave signal of the target object to obtain the power spectrum density of the brain wave signal; the power spectrum density represents the power distribution of the brain wave signal in the frequency domain; based on the frequency band energy corresponding to the α wave, the θ wave and the β wave in the brain wave signal determined by the power spectrum density, the relaxation degree of the target object is obtained; based on the frequency band range corresponding to the α wave, the θ wave and the β wave, respectively, and the frequency band energy corresponding to the first harmonic and the second harmonic of the Schumann wave resonance, the Schumann resonance energy index corresponding to the first harmonic and the second harmonic is obtained; the Schumann resonance energy index can represent the degree of influence of the Schumann wave on the brain of the target object; based on the relaxation degree and the Schumann resonance energy index, the emotion recognition result corresponding to the brain wave signal is determined.

[0105] In the above way, based on the brain's response to the Schumann resonance and combined with the degree of relaxation, the target object's emotional and psychological state can be objectively evaluated, not only focusing on the frequency characteristics of brain waves, but also considering the interaction between the brain and the natural environment, providing a more comprehensive psychological state assessment. Moreover, in the natural environment, due to the stable existence of the Schumann resonance, the target object may be more likely to achieve emotional balance and stress release through resonance with the environment, and achieve a higher level of peace. That is, when the brain's brain waves are synchronized with the Schumann resonance frequency, the individual's emotional state may be significantly improved. This resonance effect is believed to promote the activity of alpha waves, help the target object enter a deep state of relaxation, and experience inner peace and stability. Therefore, by quantifying the synchronization between the brain and the natural electromagnetic environment, and combining the performance of the Schumann resonance energy in the brain wave frequency band, the psychological state of the target object can be more accurately evaluated.

[0106] In particular, the preferred embodiments of the present application are described below in conjunction with the drawings in the specification. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present application, and are not used to limit the present application, and the embodiments of the present application and the features in the embodiments may be combined with each other if there is no conflict.

[0107] See also Figure 1As shown, it is a schematic diagram of an optional system architecture applicable to an embodiment of the present application, and the system architecture may include: a terminal device (101a, 101b) and a server 102. The terminal device (101a, 101b) and the server 102 may exchange information through a communication network, wherein the communication mode adopted by the communication network may include: a wireless communication mode and a wired communication mode. Exemplarily, the terminal device (101a, 101b) may access the network through cellular mobile communication technology to communicate with the server 102. Wherein, the cellular mobile communication technology, for example, includes the fifth generation mobile communication (5th generation mobile networks, 5G) technology or the next generation mobile communication technology. Optionally, the terminal device (101a, 101b) may access the network through a short-range wireless communication mode to communicate with the server 102. Wherein, the short-range wireless communication mode, for example, includes wireless fidelity (wireless fidelity, Wi-Fi) technology.

[0108] The embodiment of the present application does not impose any restriction on the number of communication devices involved in the above system architecture. For example, the above system architecture may include more terminal devices, or may include fewer terminal devices, or may also include other network devices. Figure 1 As shown, only the terminal devices (101a, 101b) and the server 102 are described as examples, and the above communication devices and their respective functions are briefly introduced below.

[0109] The terminal device (101a, 101b) is a device that can provide voice and / or data connectivity to users, and can be a device that supports wired and / or wireless connection.

[0110] Exemplarily, the terminal devices (101a, 101b) may include, but are not limited to: mobile phones, tablet computers, laptop computers, PDAs, mobile internet devices (MIDs), wearable devices, virtual reality (VR) devices, augmented reality (AR) devices, wireless terminal devices in industrial control, wireless terminal devices in unmanned driving, wireless terminal devices in smart grids, wireless terminal devices in transportation safety, wireless terminal devices in smart cities, or wireless terminal devices in smart homes, etc.

[0111] In addition, a related client may be installed on the terminal device (101a, 101b), and the client may be software, such as an application (APP), a browser, a short video software, etc., or a web page, a mini-program, etc.; it should be noted that the terminal device (101a, 101b) in the embodiment of the present application may enable the above-mentioned client related to brain wave-based emotion recognition to send the original signal of the target object collected in the preset multiple brain wave signal collection areas to the server 102, so as to subsequently perform method steps such as brain wave-based emotion recognition.

[0112] Server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers. It can also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms.

[0113] It is worth mentioning that the server 102 in the embodiment of the present application can perform power spectral density estimation on the EEG signal of the target object to obtain the power spectral density of the EEG signal; then, based on the frequency band energies corresponding to the α wave, θ wave and β wave in the EEG signal determined by the power spectral density, the relaxation degree of the target object is obtained; based on the frequency band ranges corresponding to the α wave, θ wave and β wave, respectively, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance, the Schumann resonance energy index corresponding to the first harmonic and the second harmonic is obtained; finally, based on the relaxation degree and the Schumann resonance energy index, emotion recognition is performed on the target object to determine the emotion recognition result that characterizes the degree of peace (or brain level harmony) of the target object.

[0114] Optionally, a pre-trained emotion recognition model can be deployed on the server 102. In this way, after obtaining the relaxation degree and Schumann resonance energy index corresponding to the EEG signal, the server 102 can input the relaxation degree and Schumann resonance energy index into the pre-trained emotion recognition model to determine the emotion recognition result of the target object. Among them, the emotion recognition model can be called a brain level and degree prediction model, and of course it can also have other names, which is not limited in the embodiments of the present application.

[0115] The above-mentioned emotion recognition model can be various network models, which is not limited in this application. For example, the above-mentioned emotion recognition model can be a convolutional neural network (CNN) model, wherein the input features of the CNN model can include relaxation degree, Schumann resonance energy index, and coefficient of variation corresponding to each frequency band, respectively, and the time series features are extracted through the convolution layer, and the emotion recognition result of the target object is output through the fully connected layer. Exemplarily, the emotion recognition result of the target object outputted above can be which kind of peaceful state or brain level harmony the target object belongs to.

[0116] The following describes the brainwave-based emotion recognition method provided by the exemplary embodiment of the present application in combination with the above-mentioned system architecture and with reference to the accompanying drawings. It should be noted that the above-mentioned system architecture is only shown to facilitate understanding of the spirit and principles of the present application, and the implementation of the present application is not limited in this respect.

[0117] See also Figure 2 As shown, it is a schematic diagram of the implementation process of an emotion recognition method based on brain waves provided in an embodiment of the present application. The execution subject takes a server as an example. The specific implementation process of the method is as follows:

[0118] S201: Estimating the power spectral density of the EEG signal of the target object to obtain the power spectral density of the EEG signal.

[0119] The power spectrum density can characterize the power distribution of the EEG signal in the frequency domain. The EEG signal can include delta wave, theta wave, alpha wave, beta wave and gamma wave.

[0120] The EEG signal of the target object can be acquired through a multi-channel EEG device. The sampling frequency of the multi-channel EEG device can be 250 Hz, that is, 250 data points are collected per second. In order to capture subtle changes in brain activity, the sampling frequency of the multi-channel EEG device can be adjusted to ensure sufficient time and frequency resolution.

[0121] Therefore, before executing step S201, the server may obtain the original signals of the target object collected in the preset multiple EEG signal collection areas, and preprocess the original signals to obtain the preprocessed original signals, and then perform artifact removal and re-reference processing on the preprocessed original signals in sequence to obtain EEG signals. Figure 3 As shown, the aforementioned preset multiple EEG signal collection areas may include: frontal pole area (Fp1, Fp2), frontal lobe area (F3, F4), central area (C3, C4) and parietal lobe area (P3, P4), etc. In this way, by covering different areas of the brain, electrical signals of different brain regions can be captured, which is helpful for analyzing the overall function of the brain and the activities of specific areas.

[0122] Optionally, the server can perform DC offset elimination and filtering on the original signal in sequence to achieve preprocessing of the original signal. For example, after the server collects the original signal through a multi-channel EEG device, it can perform DC offset elimination on the original signal to ensure signal centralization and make the signal mean of each channel zero to avoid interference from baseline offset during the analysis process, that is, to remove the baseline deviation from the original signal.

[0123] In order to remove unnecessary frequency components and noise and retain only the frequency bands related to the emotional state, the server can use a bandpass filter to retain the signal between 0.5-40 Hz and remove noise in non-target frequency bands (i.e., frequency bands corresponding to delta waves, theta waves, alpha waves, beta waves, and gamma waves). For example, the server can also use a notch filter to eliminate 50 Hz power frequency noise.

[0124] Furthermore, after obtaining the preprocessed original signal, the server may also perform blind source separation and artifact removal on the preprocessed original signal in sequence to obtain the original signal after artifact removal, so as to improve the accuracy of subsequent emotion recognition (or calmness assessment) of the target object.

[0125] Exemplarily, the server can use a preset ICA algorithm to remove artifacts from the preprocessed original signal (i.e., EEG data). It should be noted that, because the server eliminates the DC offset included in the original signal before using the preset ICA algorithm, the convergence speed and separation effect of the preset ICA algorithm can be improved.

[0126] Taking the above-mentioned preset ICA algorithm as the Fast ICA algorithm as an example, the server can use the Fast ICA algorithm to perform ICA on the preprocessed original signal. Among them, the number of independent components can be set to multiple (for example, 8) to capture the main independent signal characteristics, and the repeatability of the results can be ensured by setting random seeds; the maximum number of iterations can be set to thousands of times to ensure that the complex preprocessed original signal is fully converged. In this way, through the whitening processing of arbitrary variance, the dimensions of the mixed signal (that is, the preprocessed original signal) can be orthogonal and have unit variance, further improving the separation effect. Finally, through fitting and transformation, the multi-channel mixed signal can be separated into independent source signal matrices.

[0127] The above-mentioned artifact removal automatically detects and removes the artifact components in the original signal after blind source separation by defining an artifact detection function. By analyzing the time series of each independent component, the standard deviation is calculated and a dynamic threshold is set. If the maximum value of an independent component exceeds the threshold, it is marked as an artifact component and completely removed, reducing the interference of the artifact on the reconstructed signal, improving the purity and signal-to-noise ratio of the signal, and ensuring that the residual signal mainly contains brain-derived components. Then, through the inverse transform, the independent components after artifact removal are recombined to generate a reconstructed signal after artifact removal (that is, the original signal after artifact removal).

[0128] Then, after obtaining the original signal after the artifacts are removed, the server can also perform average re-reference on the original signal after the artifacts are removed to reduce the systematic deviation of the reference electrode and enhance the spatial resolution and specificity of the signal.

[0129] In order to improve the stability of frequency domain analysis of EEG signals, in an optional implementation, when executing step S201, the server can divide the EEG signal into multiple continuous sub-EEG signals based on the preset signal duration, thereby respectively estimating the power spectral density of the multiple sub-EEG signals, obtaining multiple sub-power spectral densities, and obtaining the power spectral density based on the multiple sub-power spectral densities. Among them, there is an overlapping area in the signal time corresponding to any two adjacent sub-EEG signals in the aforementioned multiple sub-EEG signals. Taking the preset signal duration of 1 second as an example, the server can divide the 4-second EEG signal into 8 1-second sub-EEG signals, and there is a 50% time overlap area between the sub-EEG signals.

[0130] It should be understood that each of the above-mentioned sub-power spectral densities, that is, the power spectral density corresponding to the corresponding sub-EEG signal, that is, the power spectral density corresponding to the sub-EEG signal can also have other names, and the embodiments of the present application are not limited to this.

[0131] Exemplarily, the server can use the Welch method to perform spectrum estimation on the segmented sub-EEG signals. Still taking a 4-second EEG signal as an example, first, the 4-second EEG signal segment can be divided into 8 1-second sub-EEG signals, with a 50% time overlap between the sub-EEG signals to increase the stability of the spectrum estimation. Then, the server can perform windowing processing on each sub-EEG signal by using a windowing method such as a Hanning window to reduce spectrum leakage.

[0132] Based on the above method, this overlapping sliding window method not only ensures the signal continuity between multiple sub-EEG signals, but also increases the data points to improve the stability of frequency domain analysis, which is suitable for continuous emotion monitoring scenarios.

[0133] Then, the server can perform frequency domain transformation on each windowed sub-EEG signal through frequency domain transformation methods such as fast Fourier transform (FFT) to convert the time domain signal into frequency domain representation. The calculation formula corresponding to the aforementioned FFT can be specifically expressed as follows:

[0134]

[0135] in, X [ k ] represents the first k Quantity, x [ n ] represents the time domain representation of the sub-EEG signal, N Indicates the signal length of the sub-EEG signal.

[0136] Furthermore, the server can calculate the sub-power spectrum density of each sub-EEG signal, and sum and average the sub-power spectrum densities corresponding to multiple sub-EEG signals to obtain the Welch power spectrum estimation (i.e., the average power spectrum density of multiple sub-EEG signals is used as the power spectrum density of the EEG signal). Optionally, the calculation formula of the aforementioned Welch power spectrum estimation is as follows:

[0137]

[0138] in, P Welch [ f ] represents the power spectrum density corresponding to the EEG signal, M Indicates the number of sub-EEG signals corresponding to each EEG signal, P k [ f ] represents the sub-power spectral density corresponding to each sub-EEG signal.

[0139] S202: Obtaining the relaxation degree of the target object based on the frequency band energies corresponding to the α wave, the θ wave and the β wave in the EEG signal determined by the power spectrum density.

[0140] Exemplarily, when executing step S202, after obtaining the power spectrum density corresponding to the EEG signal, the server can determine the frequency band energy corresponding to the α wave, the θ wave and the β wave according to the frequency band ranges corresponding to the α wave, the θ wave and the β wave. The frequency band range corresponding to the α wave can be: 8-13 Hz, the frequency band range corresponding to the θ wave can be: 4-8 Hz, and the frequency band range corresponding to the β wave can be: 13-30 Hz. The calculation formula for the frequency band energy corresponding to each frequency band range is as follows:

[0141]

[0142] in,E band Indicates the frequency band energy corresponding to the corresponding frequency band range ，P Welch [ f ] represents the power spectrum density corresponding to the EEG signal, f high Indicates the upper frequency limit of the corresponding frequency band range. For example, the upper frequency limit of the frequency band range corresponding to theta wave is 8 Hz. f low Indicates the lower frequency limit of the aforementioned corresponding frequency band range. For example, the lower frequency limit of the frequency band range corresponding to the θ wave is 4 Hz.

[0143] It should be understood that the frequency ranges corresponding to the above-mentioned α wave, θ wave and β wave are only examples and are not intended to be specific limitations. For example, the frequency range corresponding to the α wave may also be 7-12 Hz or 8-12 Hz, the frequency range corresponding to the θ wave may also be 3-7 Hz or 4-7 Hz, and the frequency range corresponding to the β wave may also be 12-30 Hz.

[0144] In an optional implementation, the server can obtain the above relaxation degree based on the frequency band energy corresponding to the α wave and the frequency band energy corresponding to the θ wave and the β wave in the EEG signal adjacent to the frequency band range of the α wave. For example, the above relaxation degree can be the ratio of the frequency band energy corresponding to the α wave to the sum of the frequency band energy corresponding to the θ wave and the β wave. The calculation formula of the above relaxation degree can be specifically expressed as follows:

[0145]

[0146] in, R represents the degree of relaxation (or relaxation index), E α Indicates the frequency band energy corresponding to the α wave, E θ Indicates the frequency band energy corresponding to theta waves, E β Indicates the frequency band energy corresponding to β waves.

[0147] Since alpha waves can represent the target object's awake and relaxed state, the relative increase in the energy of the frequency band corresponding to alpha waves indicates that the target object is more relaxed, that is, the target object is more peaceful. Therefore, the degree of relaxation can represent the proportion of emotions dominated by alpha waves in the target object's emotions under the combined effect of multiple brain waves.

[0148] S203: Based on the frequency band ranges corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance, the Schumann resonance energy index corresponding to the first harmonic and the second harmonic is obtained.

[0149] The Schumann resonance energy index can characterize the degree of influence of the Schumann wave on the target object's brain. The frequency of the first harmonic is the same as the main frequency of the Schumann wave, so the first harmonic can also be called the fundamental wave that resonates with the Schumann wave. The frequency of the second harmonic is usually twice the main frequency of the Schumann wave.

[0150] In actual measurement, the frequency of the Schumann resonance (i.e., the frequency corresponding to the first harmonic and / or the frequency of the second harmonic) may fluctuate due to individual physiological differences of the target object, environmental noise, device resolution, etc. In order to ensure that the relevant Schumann resonance energy is fully contained, an appropriate frequency band range can be selected near the fundamental frequency (i.e., the frequency of the first harmonic) and the harmonic frequency (i.e., the frequency of the second harmonic). That is, the frequency band range is set separately for the first harmonic and the second harmonic.

[0151] For example, with 7.83 Hz as the center frequency, the frequency band is extended by 0.5 Hz upward and downward to form a frequency band with a total bandwidth of 1 Hz, that is, 7.33 Hz to 8.33 Hz is used as the frequency band corresponding to the first harmonic. Similarly, with 14.3 Hz as the center frequency, the frequency band is extended by 0.5 Hz upward and downward to form a frequency band with a total bandwidth of 1 Hz, that is, 13.8 Hz to 14.8 Hz is used as the frequency band corresponding to the second harmonic. This can cover the slight changes in frequency caused by environmental and individual differences, and also ensure that the extracted frequency band energy is related to the fundamental frequency of the Schumann resonance.

[0152] Optionally, the frequency band corresponding to the first harmonic may also be referred to as Schumann resonance frequency band 1, and the frequency band corresponding to the first harmonic may also be referred to as Schumann resonance frequency band 2. Of course, the frequency band corresponding to the first harmonic and / or the frequency band corresponding to the second harmonic may also have other names, which are not specifically limited in the embodiments of the present application.

[0153] The frequency band range corresponding to the above-mentioned first harmonic and the frequency band range corresponding to the above-mentioned second harmonic have frequency overlap with the frequency band ranges corresponding to α wave, θ wave and β wave respectively, which can better capture the coupling relationship between Schumann resonance wave (i.e., the first harmonic and the second harmonic) and the brain wave frequency band, that is, it can better capture the resonance effect related to brain electrical activity, and then make full use of the potential regulatory effect of natural resonance on the emotional state of the brain.

[0154] The third harmonic (e.g., the frequency of the third harmonic is 20.3 Hz) and its higher harmonics generated by the Schumann wave resonance are also components of the Schumann resonance, but due to their higher frequencies, they overlap less with the frequency bands corresponding to the alpha and beta waves that are mainly involved in relaxation (or peace) and emotional regulation. Therefore, the third harmonic and its higher harmonics generated by the Schumann wave resonance can be temporarily ignored to perform emotional recognition on the target object's peace.

[0155] Therefore, in an optional implementation, see Figure 4 As shown, it is a schematic diagram of an implementation process of a method for obtaining the Schumann resonance energy index provided in an embodiment of the present application. The specific implementation process of the method is as follows:

[0156] S401: Determine frequency band energies corresponding to the first harmonic and the second harmonic respectively based on the frequency band ranges set for the first harmonic and the second harmonic respectively.

[0157] It should be understood that the manner of determining the frequency band energies corresponding to the first harmonic and the second harmonic respectively may be the same as or different from the manner of determining the frequency band energies corresponding to the α wave, the θ wave, and the β wave respectively described above, and the embodiments of the present application do not specifically limit this. For example, a specific frequency band energy collection device is used to collect the frequency band energy of the first harmonic within the frequency band set for the first harmonic, and to collect the frequency band energy of the second harmonic within the frequency band set for the second harmonic.

[0158] S402: Based on the frequency band ranges corresponding to the first harmonic and the second harmonic, and the frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively, determine the sub-frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively.

[0159] Optionally, the sum of the sub-frequency band range corresponding to the α wave and the sub-frequency band range corresponding to the θ wave includes the frequency band range corresponding to the first harmonic, and the sub-frequency band range corresponding to the β wave includes the frequency band range corresponding to the second harmonic. Exemplarily, the sub-frequency band range corresponding to the α wave may be the low frequency band of the α wave (e.g., 8-10 Hz), the sub-frequency band range corresponding to the θ wave may be the high frequency band of the θ wave (e.g., 6-8 Hz), and the sub-frequency band range corresponding to the β wave may be the low frequency band of the β wave (e.g., 12-20 Hz). The frequency band range corresponding to the first harmonic is: 7.33 Hz to 8.33 Hz, and the frequency band range corresponding to the second harmonic is: 13.8 Hz to 14.8 Hz.

[0160] S403: Based on the sub-frequency band ranges and power spectrum densities corresponding to the α wave, the θ wave and the β wave, respectively, determine the sub-frequency band energies corresponding to the α wave, the θ wave and the β wave, respectively.

[0161] For example, the sub-band energy corresponding to the above α wave can be expressed as Bαs , the sub-frequency range corresponding to the above θ wave can be expressed as B θs , the sub-frequency range corresponding to the above β wave can be expressed as B βs The sub-band energy corresponding to the above α wave can be expressed as E αs , the sub-band energy corresponding to the above θ wave can be expressed as E θs , the sub-band energy corresponding to the above β wave can be expressed as E βs .

[0162] S404: Obtaining a Schumann resonance energy index based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic.

[0163] In an optional implementation, when executing step S404, the server can obtain a first energy ratio based on the sum of the sub-band energies corresponding to the α wave and the sub-band energies corresponding to the θ wave, and the band energy corresponding to the first harmonic, that is, the ratio of the band energy corresponding to the first harmonic to the sum of the energies of the sub-band energies corresponding to the α wave and the sub-band energies corresponding to the θ wave as the first energy ratio; and obtain a second energy ratio based on the sub-band energy corresponding to the β wave and the band energy corresponding to the second harmonic, that is, the ratio of the band energy corresponding to the second harmonic to the sub-band energy corresponding to the β wave as the second energy ratio; finally, the Schumann resonance energy index is obtained based on the first energy ratio and the second energy ratio.

[0164] Exemplarily, the calculation formula of the above-mentioned Schumann resonance energy index can be specifically expressed as follows:

[0165]

[0166] in, SREI Represents the Schumann resonance energy index corresponding to the first and second harmonics, E schumann1 Indicates the frequency band energy corresponding to the first harmonic, E schumann2 Indicates the frequency band energy corresponding to the second harmonic, E θs represents the sub-band energy corresponding to theta wave, E αs represents the sub-band energy corresponding to the α wave, E βs represents the sub-band energy corresponding to the β wave, ω0 and ω1 are weight coefficients, ω0+ω1=1, and the specific values ​​of ω0 and ω1 can be determined according to actual conditions, which is not limited in the present embodiment.E schumann1 / ( E θs + E αs ) is the first energy ratio, E schumann2 / E βs is the second energy ratio.

[0167] Optionally, the sub-band energy corresponding to the above theta wave E θs Specifically, it can be the sub-band energy corresponding to the high frequency band of theta wave (e.g., 6-8 Hz), for example, it can be expressed as E θhigh The sub-band energy corresponding to the above α wave E αs Specifically, it can be the sub-band energy corresponding to the low frequency band of the α wave (e.g., 8-10 Hz), for example, it can be expressed as E αlow The sub-band energy corresponding to the above β wave E βs Specifically, it can be the sub-band energy corresponding to the low frequency band of the β wave (e.g., 13-20 Hz), for example, it can be expressed as E βlow .

[0168] S204: Determine the emotion recognition result corresponding to the EEG signal based on the relaxation degree and the Schumann resonance energy index.

[0169] The above emotion recognition result can characterize the target object's peacefulness or brain level harmony, so the above emotion recognition result can also be called a classification result of the peacefulness (or brain level harmony). Exemplarily, the above emotion recognition result can be any one of the three recognition results of "very peaceful", "relatively peaceful" and "unpeaceful".

[0170] Therefore, in an optional implementation, see Figure 5 As shown, when executing step S204, the server can input the relaxation degree and the Schumann resonance energy index into a pre-trained emotion recognition model to obtain the prediction probabilities corresponding to a variety of recognition results (for example, "very peaceful", "relatively peaceful" and "unpeaceful"), thereby determining the emotion recognition result corresponding to the EEG signal based on the prediction probabilities corresponding to the aforementioned multiple recognition results.

[0171] Exemplarily, the server may use the recognition result corresponding to the maximum predicted probability among the predicted probabilities corresponding to the above-mentioned multiple recognition results as the emotion recognition result corresponding to the EEG signal. Still taking the three recognition results of "very peaceful", "relatively peaceful" and "unpeaceful" as examples. Assuming that the predicted probabilities corresponding to "very peaceful", "relatively peaceful" and "unpeaceful" are 0.118, 0.752 and 0.13 respectively, the server may use the recognition result with the maximum predicted probability (i.e. "relatively peaceful") as the emotion recognition result corresponding to the EEG signal.

[0172] In order to further improve the accuracy of emotion recognition. In an optional implementation, see Figure 6 As shown, the server can also determine the coefficient of variation corresponding to the α wave, the θ wave and the β wave based on the frequency band energy corresponding to the α wave, the θ wave and the β wave, and then perform emotion recognition on the target object based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to the α wave, the θ wave and the β wave. The calculation formula of the coefficient of variation is as follows:

[0173]

[0174] in, CV band Indicates the coefficient of variation corresponding to the frequency band range. σ band Indicates the standard deviation of the frequency band energy corresponding to the aforementioned corresponding frequency band range, μ band Represents the mean value of the frequency band energy corresponding to the aforementioned corresponding frequency band range.

[0175] Based on the above method, since the coefficient of variation can reflect the relative discreteness of the data, the coefficient of variation corresponding to the above α wave, θ wave and β wave respectively can capture the volatility of the frequency band energy corresponding to the α wave, θ wave and β wave respectively over time, and can more accurately judge the target object's calmness or brain level and degree, further improving the accuracy of emotion recognition.

[0176] Furthermore, the server can input the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α waves, θ waves and β waves respectively into a pre-trained emotion recognition model to obtain the prediction probabilities corresponding to multiple recognition results, thereby determining the emotion recognition results of the target object based on the prediction probabilities corresponding to the multiple recognition results.

[0177] For example, see Figure 7As shown, it is a specific schematic diagram of the correlation between the characteristics of a model evaluation provided by an embodiment of the present application and the degree of peace of the target object. The coefficient of variation corresponding to the α wave and the coefficient of variation corresponding to the θ wave are negatively correlated with the degree of peace of the target object, which can be: -0.46 and -0.46 respectively. The coefficient of variation corresponding to the β wave, the Schumann resonance energy index and the degree of relaxation are negatively correlated with the degree of peace of the target object, which can be: 0.46, 0.89 and 0.89 respectively. The correlation between different features can also be as follows Figure 7 As shown, no further description is given here.

[0178] In an optional implementation, the input layer of the above-mentioned emotion recognition model can be 5 related features including 8 channels, namely, relaxation, Schumann resonance energy index, and coefficients of variation corresponding to alpha waves, theta waves, and beta waves, respectively. Each sample used to train the emotion recognition model can be 2 seconds of EEG data, and the sample size of the input layer can be 8×500×5. The label of each sample corresponds to the classification label of the peaceful state. The convolution layer can include two one-dimensional convolution layers, respectively including 32 and 64 convolution kernels of size 1×5, and the convolution step size is 1. Through the convolution operation, the emotion recognition model can extract key feature patterns in the local time series in the time dimension. Each convolution layer can be followed by a maximum pooling layer of size 1×2 to gradually reduce the feature dimension and prevent overfitting. The feature data is also batch normalized to enhance the stability of the model, and a preset first activation function (e.g., ReLU activation function) is used to increase the nonlinear expression ability of the model. The features processed by the convolution and pooling layers are input into the fully connected layer, and the preset second activation function (such as the Softmax activation function) is used to complete the classification mapping of the emotion results. The fully connected layer can contain three neurons, corresponding to the three peaceful states of "very peaceful", "relatively peaceful" and "unpeaceful", that is, each neuron is used to output the predicted probability of the corresponding peaceful state. The output of the emotion recognition model is the predicted probability corresponding to the three peaceful states for each sample.

[0179] In the training phase of the emotion recognition model, the training data (i.e., training samples) can be feature extracted to obtain the corresponding eigenvalue data, and these eigenvalue data can be input into the emotion recognition model for training. The emotion recognition model can use methods such as the cross entropy loss function to measure the classification error, and update the network weights through the back propagation algorithm, so that the emotion recognition model can better predict the target object's peaceful state after each round of training. Among them, each training data corresponds to a peaceful state label, and the training data can be divided into a training set, a validation set, and a test set, and the leave-one-out method is used for division to improve the generalization ability of the model. In order to further improve the stability of training, the output of the convolutional layer will be batch normalized to reduce internal covariate shift and accelerate model convergence. In addition, the dropout technology is applied to the fully connected layer to randomly discard some neurons, reduce the excessive dependence of the emotion recognition model on certain features, and effectively reduce the risk of overfitting.

[0180] During the training process, the performance of the emotion recognition model is evaluated on the validation set, and the prediction performance of the model is ensured by calculating indicators such as accuracy and recall. After the training is completed, the weights and parameters of the optimal model are saved for use in the subsequent application stage. That is, based on a large amount of eigenvalue data and its corresponding peace state labels, the data is processed in batches and divided into training sets, validation sets, and test sets using the leave-one-out method. The preset first activation function is used to activate the function, and the risk of overfitting is reduced by maximum pooling. Finally, the model parameters are optimized in the back propagation process through cross entropy loss. After the training is completed, the model will save the final weights and parameters for inference in the subsequent application stage.

[0181] Therefore, based on the emotion recognition method described in steps S201 to S204, refer to Figure 8 As shown, the server can obtain the original EEG signal of the target object in real time, and obtain the target EEG signal by performing data preprocessing (such as DC offset elimination and filtering), artifact removal and re-reference processing on the original EEG signal in sequence. Then, the aforementioned target EEG signal is subjected to feature extraction for training the emotion recognition model, or input into the emotion recognition model to complete the emotion recognition for the target EEG signal and obtain the emotion recognition result, that is, to complete the evaluation and classification of the target object's peacefulness.

[0182] In summary, in the resource scheduling method of the example provided in the embodiment of the present application, after the power spectrum density of the EEG signal of the target object is estimated and the power spectrum density of the EEG signal is obtained, the frequency band energy corresponding to the α wave, the θ wave and the β wave that can reflect the target object's emotional peace state can be determined, and the frequency band energy corresponding to the first harmonic and the second harmonic generated by the Schumann wave resonance that can affect the target object's emotions can be determined. In this way, the relaxation degree obtained by the frequency band energy corresponding to the α wave, the θ wave and the β wave, and the Schumann resonance energy index obtained based on the frequency band range corresponding to the α wave, the θ wave and the β wave, and the frequency band energy corresponding to the first harmonic and the second harmonic of the Schumann resonance, can be used to accurately identify the target object's peaceful state, thereby improving the accuracy of emotion recognition.

[0183] Further, based on the same technical concept, the embodiment of the present application provides an emotion recognition device based on brain waves, and the emotion recognition device based on brain waves is used to implement the above method flow of the embodiment of the present application. Fig. 9 As shown, the brain wave-based emotion recognition device 900 may include: a signal estimation module 901, a first processing module 902, a second processing module 903 and an emotion recognition module 904, wherein:

[0184] The signal estimation module 901 is used to estimate the power spectrum density of the EEG signal of the target object to obtain the power spectrum density of the EEG signal; the power spectrum density represents the power distribution of the EEG signal in the frequency domain;

[0185] The first processing module 902 is used to obtain the relaxation degree of the target object based on the frequency band energy corresponding to the alpha wave, theta wave and beta wave in the EEG signal determined by the power spectrum density;

[0186] The second processing module 903 is used to obtain the Schumann resonance energy index corresponding to the first harmonic and the second harmonic based on the frequency band ranges corresponding to the α wave, the θ wave and the β wave respectively, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance respectively; the Schumann resonance energy index represents the degree of influence of the Schumann wave on the brain of the target object;

[0187] The emotion recognition module 904 is used to determine the emotion recognition result corresponding to the EEG signal based on the relaxation degree and the Schumann resonance energy index.

[0188] In an optional embodiment, before performing power spectrum density estimation on the EEG signal of the target object to obtain the power spectrum density, the signal estimation module 901 is further used to:

[0189] Acquire the original signals of the target object collected in the preset multiple EEG signal collection areas, and pre-process the original signals to obtain the pre-processed original signals;

[0190] The preprocessed original signal is subjected to artifact removal and re-reference processing in sequence to obtain the EEG signal.

[0191] In an optional embodiment, when performing artifact removal and re-reference processing on the pre-processed original signal in sequence to obtain the EEG signal, the signal estimation module 901 is specifically used to:

[0192] The preprocessed original signal is subjected to blind source separation and artifact removal processing in sequence to obtain the original signal after artifact removal;

[0193] The original signal after artifact removal is re-referenced to obtain the EEG signal.

[0194] In an optional embodiment, when obtaining the Schumann resonance energy index corresponding to the first harmonic and the second harmonic based on the frequency band ranges corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic of the Schumann resonance, the first processing module 902 is specifically used to:

[0195] Determine frequency band energies corresponding to the first harmonic and the second harmonic respectively based on the frequency band ranges set for the first harmonic and the second harmonic respectively;

[0196] Based on the frequency band ranges corresponding to the first harmonic and the second harmonic, and the frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively, determining the sub-frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively;

[0197] Based on the sub-frequency band ranges and power spectrum densities corresponding to the α wave, the θ wave and the β wave, respectively, the sub-frequency band energies corresponding to the α wave, the θ wave and the β wave, respectively, are determined;

[0198] The Schumann resonance energy index is obtained based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic.

[0199] In an optional embodiment, when obtaining the Schumann resonance energy index based on the sub-band energies corresponding to the α wave, the θ wave and the β wave, and the frequency band energies corresponding to the first harmonic and the second harmonic, the second processing module 903 is specifically used to:

[0200] The ratio of the frequency band energy corresponding to the first harmonic to the sum of the energy of the sub-frequency band corresponding to the α wave and the energy of the sub-frequency band corresponding to the θ wave is taken as the first energy ratio;

[0201] The ratio of the frequency band energy corresponding to the second harmonic to the sub-frequency band energy corresponding to the β wave is used as the second energy ratio;

[0202] A Schumann resonance energy index is obtained based on the first energy ratio and the second energy ratio.

[0203] In an optional embodiment, the second processing module 903 is further configured to:

[0204] Determine the coefficient of variation of the frequency band energy corresponding to alpha waves, theta waves and beta waves respectively;

[0205] When determining the emotion recognition result corresponding to the EEG signal based on the relaxation degree and the Schumann resonance energy index, the emotion recognition module 904 is specifically used to:

[0206] The emotion recognition results are determined based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α waves, θ waves and β waves respectively.

[0207] In an optional embodiment, when determining the emotion recognition result based on the coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to the α wave, the θ wave and the β wave respectively, the emotion recognition module 904 is specifically used to:

[0208] The coefficient of variation, relaxation degree and Schumann resonance energy index corresponding to α wave, θ wave and β wave are input into the pre-trained emotion recognition model to obtain the prediction probabilities corresponding to various recognition results.

[0209] The emotion recognition result is determined based on the prediction probabilities corresponding to the various recognition results.

[0210] Based on the description of the above method embodiment and device embodiment, the exemplary embodiment of the present invention further provides an electronic device, including: at least one processor; and a memory connected to the at least one processor in communication. The memory stores a computer program that can be executed by the at least one processor, and the computer program is used to enable the electronic device to perform the method according to the embodiment of the present invention when executed by the at least one processor.

[0211] An embodiment of the present application also provides a non-transitory computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0212] An embodiment of the present application also provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method according to an embodiment of the present application.

[0213] See also Fig.10As shown, the structured block diagram of the electronic device 1000 that can be used as the server or client of the present application will now be described, which is an example of the hardware device that can be applied to various aspects of the present application. The electronic device is intended to represent the computer device of various forms of digital electronics, such as, laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, personal digital processing, cellular phones, smart phones, wearable devices and other similar computing devices. The components shown herein, their connections and relationships, and their functions are only used as examples, and are not intended to limit the implementation of the present application described herein and / or required.

[0214] like Fig.10 As shown, the electronic device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 to a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0215] A plurality of components in the electronic device 1000 are connected to the I / O interface 1005, including: an input unit 1006, an output unit 1007, a storage unit 1008, and a communication unit 1009. The input unit 1006 may be any type of device capable of inputting information to the electronic device 1000, and the input unit 1006 may receive input digital or character information, and generate key signal inputs related to user settings and / or function control of the electronic device. The output unit 1007 may be any type of device capable of presenting information, and may include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 1008 may include but is not limited to a disk, an optical disk. The communication unit 1009 allows the electronic device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and may include but is not limited to a modem, a network card, an infrared communication device, a wireless communication transceiver, and / or a chipset, such as a Bluetooth device, a WiFi device, a worldwide interoperability for microwave access (WiMax) device, a cellular communication device, and / or the like.

[0216] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 1001 performs the various methods and processes described above. For example, in some embodiments, the above-mentioned brain wave-based emotion recognition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1000 via the ROM 1002 and / or the communication unit 1009. In some embodiments, the computing unit 1001 may be configured to perform the above-mentioned brain wave-based emotion recognition method in any other appropriate manner (e.g., by means of firmware).

[0217] The program code for implementing the method of the present application can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code can be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0218] In the context of the present application, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a RAM, a ROM, an erasable programmable read-only memory (EPROM) or a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0219] As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0220] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a cathode ray tub (CRT) or a liquid crystal display (LCD) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0221] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0222] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0223] Furthermore, it should be understood that what is disclosed above is only a preferred embodiment of the present application, and certainly cannot be used to limit the scope of rights of the present invention. Therefore, equivalent changes made according to the claims of the present invention are still within the scope covered by the present application.

Claims

1. A method for emotion recognition based on brain waves, characterized in that: include: Performing power spectral density estimation on the EEG signal of the target object to obtain the power spectral density of the EEG signal; the power spectral density represents the power distribution of the EEG signal in the frequency domain; Obtaining the relaxation degree of the target object based on the frequency band energies corresponding to the alpha wave, theta wave and beta wave in the EEG signal respectively determined by the power spectrum density; Based on the frequency band ranges respectively set for the first harmonic and the second harmonic of the Schumann resonance, determining the frequency band energies respectively corresponding to the first harmonic and the second harmonic; Based on the frequency band ranges corresponding to the first harmonic and the second harmonic, and the frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively, determining the sub-frequency band ranges corresponding to the α wave, the θ wave, and the β wave, respectively; Determine the sub-frequency band energies corresponding to the α wave, the θ wave and the β wave respectively based on the sub-frequency band ranges corresponding to the α wave, the θ wave and the β wave respectively, and the power spectrum density; The ratio of the frequency band energy corresponding to the first harmonic to the sum of the energy of the sub-frequency band corresponding to the α wave and the energy of the sub-frequency bands corresponding to the θ wave is taken as the first energy ratio; The ratio of the frequency band energy corresponding to the second harmonic to the sub-frequency band energy corresponding to the β wave is used to obtain a second energy ratio; Based on the first energy ratio and the second energy ratio, obtaining the Schumann resonance energy index; The Schumann resonance energy index represents the degree of influence of the Schumann wave on the brain of the target object; Based on the relaxation degree and the Schumann resonance energy index, an emotion recognition result corresponding to the electroencephalogram signal is determined.

2. The method according to claim 1, characterized in that Before estimating the power spectrum density of the EEG signal of the target object and obtaining the power spectrum density, the method further includes: Acquire the original signals of the target object collected in a plurality of preset EEG signal collection areas, and preprocess the original signals to obtain the preprocessed original signals; Artifact removal and re-reference processing are performed in sequence on the pre-processed original signal to obtain the electroencephalogram signal.

3. The method according to claim 2, characterized in that Performing artifact removal and re-reference processing on the pre-processed original signal in sequence to obtain the EEG signal, including: Performing blind source separation and artifact removal processing on the preprocessed original signal in sequence to obtain the original signal after artifact removal; The original signal after the artifact removal is subjected to re-reference processing to obtain the electroencephalogram signal.

4. The method according to claim 1, characterized in that The sum of the sub-frequency band range corresponding to the α wave and the sub-frequency band range corresponding to the θ wave includes the frequency band range corresponding to the first harmonic, and the sub-frequency band range corresponding to the β wave includes the frequency band range corresponding to the second harmonic.

5. The method according to any one of claims 1 to 4, characterized in that The method further comprises: Determining the coefficient of variation of the frequency band energy corresponding to the α wave, the θ wave and the β wave respectively; Then, based on the relaxation degree and the Schumann resonance energy index, determining the emotion recognition result corresponding to the EEG signal further includes: The emotion recognition result is determined based on the coefficients of variation respectively corresponding to the alpha wave, the theta wave and the beta wave, the degree of relaxation and the Schumann resonance energy index.

6. The method according to claim 5, characterized in that The determining of the emotion recognition result based on the coefficient of variation respectively corresponding to the α wave, the θ wave and the β wave, the relaxation degree and the Schumann resonance energy index comprises: Inputting the coefficient of variation, the degree of relaxation and the Schumann resonance energy index respectively corresponding to the alpha wave, the theta wave and the beta wave into a pre-trained emotion recognition model to obtain prediction probabilities respectively corresponding to a plurality of recognition results; The emotion recognition result is determined based on the prediction probabilities respectively corresponding to the multiple recognition results.

7. An emotion recognition device based on brain waves, characterized in that: include: A signal estimation module, used to estimate the power spectrum density of the EEG signal of the target object to obtain the power spectrum density of the EEG signal; The power spectral density represents the power distribution of the EEG signal in the frequency domain; A first processing module is used to obtain the relaxation degree of the target object based on the frequency band energies corresponding to the alpha wave, theta wave and beta wave in the EEG signal determined by the power spectrum density; a second processing module, for determining frequency band energies corresponding to the first harmonic and the second harmonic respectively based on the frequency band ranges respectively set for the first harmonic and the second harmonic of the Schumann resonance; determining sub-frequency band ranges corresponding to the α wave, the θ wave and the β wave respectively based on the frequency band ranges respectively corresponding to the first harmonic and the second harmonic, and the frequency band ranges respectively corresponding to the α wave, the θ wave and the β wave; determining sub-frequency band energies corresponding to the α wave, the θ wave and the β wave respectively based on the sub-frequency band ranges respectively corresponding to the α wave, the θ wave and the β wave, and the power spectrum density; taking the ratio of the frequency band energy corresponding to the first harmonic to the sum of the energies of the sub-frequency band energy corresponding to the α wave and the sub-frequency band energy corresponding to the θ wave as the first energy ratio; The ratio of the frequency band energy corresponding to the second harmonic to the sub-frequency band energy corresponding to the β wave is used to obtain a second energy ratio; Based on the first energy ratio and the second energy ratio, obtaining the Schumann resonance energy index; The Schumann resonance energy index represents the degree of influence of the Schumann wave on the brain of the target object; The emotion recognition module is used to determine the emotion recognition result corresponding to the EEG signal based on the relaxation degree and the Schumann resonance energy index.

8. An electronic device, comprising: processor; as well as Memory for storing programs, The program includes instructions, which, when executed by the processor, cause the processor to perform the method according to any one of claims 1 to 6.

Images