Emotion recognition dataset construction method and system based on brain wave mapping values and application thereof

Abstract

This invention relates to the field of electroencephalography (EEG) technology, and provides a method, system, and application for constructing an emotion recognition dataset based on EEG mapping values. The method includes: S1, collecting EEG mapping values ​​through multiple sampling points; S2, separating the alpha, beta, and gamma waves of the EEG based on the collected EEG mapping values, and determining whether each sampling interval represents a first, second, or third emotional state according to a set criterion based on the values ​​of the alpha, beta, and gamma waves; S3, collecting EEG mapping values ​​according to different emotional states, and removing outliers from the collected data for each emotional state to obtain an EEG emotion recognition dataset. The method provided by this invention offers a solution for removing outliers from EEG waves of different emotions, solving the problems of difficulty in constructing EEG emotion recognition datasets and the presence of numerous outliers.

Description

Technical Field

[0001] This invention relates to the field of brainwave technology, and in particular to a method, system and application for constructing an emotion recognition dataset based on brainwave mapping values. Background Technology

[0002] Human emotions are closely related to brain activity, and brain electrical activity can be monitored through electroencephalography (EEG). EEG reflects the physiological state of the brain, and different emotional states induce different patterns of brain electrical activity. Emotion recognition is crucial in many applications, such as medical health monitoring, human-computer interaction, education and training, and security assessment. However, existing EEG datasets contain a large number of outliers, which can bias machine learning and deep learning models, affecting their generalization ability and the interpretability of the results.

[0003] To address the aforementioned problems, this invention proposes a method for constructing a brainwave emotion recognition dataset that automatically removes abnormal brainwave values ​​and builds a high-quality dataset. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a method, system and application for constructing an emotion recognition dataset based on EEG mapping values, which automatically removes outliers from EEG data and constructs a high-quality dataset.

[0005] The present invention adopts the following technical solution:

[0006] On one hand, this invention provides a method for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values, including:

[0007] S1. Collect EEG mapping values ​​through multiple sampling points;

[0008] S2. Based on the collected EEG mapping values, separate the alpha, beta, and gamma waves of the EEG, and determine whether the sampling interval is the first emotional state, the second emotional state, or the third emotional state according to the values ​​of the alpha, beta, and gamma waves in each sampling interval and the set criteria.

[0009] S3. Collect EEG mapping values ​​according to different emotional states, and remove abnormal data from the collected data for each emotional state to obtain an EEG emotion recognition dataset.

[0010] In addition to the aspects and any possible implementations described above, a further implementation is provided in which, in step S2, multiple sampling points are arranged in chronological order, with each m1 point forming a sampling interval, for a total of n sampling intervals;

[0011] The criteria for determining a sampling interval as the second emotional state are as follows: if there are m2 or more first peak points in the beta wave within the sampling interval, then the sampling interval is determined to be the second emotional state; the first peak point is a sampling point whose beta wave mapping value is greater than the mean of the beta wave mapping values ​​of all sampling points in the interval by more than three standard deviations.

[0012] The criteria for determining a sampling interval as the third emotional state are as follows: if there are m3 or more second peak points of the gamma wave within the sampling interval, then the sampling interval is determined to be the third emotional state; the second peak point is a sampling point where the gamma wave mapping value is greater than the mean of the gamma wave mapping values ​​of all sampling points in the interval by more than three standard deviations.

[0013] The criteria for determining the sampling interval as the first emotional state are: if the alpha wave, beta wave, and gamma wave are relatively stable within the sampling interval and have fewer than m4 peak points, and m4≤min(m2,m3), then the sampling interval is determined to be the first emotional state.

[0014] n, m1, m2, m3, and m4 are all natural numbers. The ratios of m2 to m1 and m3 to m1 are determined through experiments.

[0015] In addition to the aspects and any possible implementations described above, a further implementation is provided in which, in step S2, the sampling time interval is 1 second, the total number of sampling points is 360, and every 10 sampling points constitute a sampling interval, for a total of 36 sampling intervals;

[0016] The criterion for determining a sampling interval as the second emotional state is: if the beta wave has 3 or more peak points within the sampling interval, then the sampling interval is determined to be the second emotional state.

[0017] The criterion for determining a sampling interval as the third emotional state is: if the gamma wave has 4 or more peak points within the sampling interval, then the sampling interval is determined to be the third emotional state.

[0018] The criteria for determining the first emotional state in a sampling interval are: if the alpha, beta, and gamma waves are relatively stable within the sampling interval and each has fewer than 3 peak points, then the sampling interval is determined to be the first emotional state.

[0019] In addition to the aspects and any possible implementations described above, a further implementation is provided in which, in step S1, the original brainwaves of the test subject are collected using an electroencephalogram (EEG) sensor, and the original brainwaves are locally amplified and characterized by EEG mapping values ​​to avoid the phenomenon that the original brainwaves have indistinct features due to their small values.

[0020] In addition to the aspects and any possible implementations described above, a further implementation is provided in which, in step S3, abnormal mapping values ​​in the first emotional state data are automatically removed using box plot rules, and abnormal mapping values ​​in the second and third emotional state data are automatically removed using a sliding window local extremum filtering method.

[0021] As described above, and considering any possible implementation methods, a further implementation method is provided: The specific method for removing abnormal mapping values ​​is as follows:

[0022] All EEG mapping values ​​for the first emotional state are arranged from smallest to largest to obtain the first and third quartiles of each type of alpha, beta, and gamma wave. The IQR (Interquartile Range) is obtained by subtracting the first quartile from the third quartile. The lower bound is the IQR value located at 1.5 times the first quartile, and the upper bound is the IQR value located at 1.5 times the third quartile. Data outside the upper and lower bounds are considered abnormal EEG waves in the first emotional state data.

[0023] A sliding window was set for the β wave in the second emotional state. The maximum value in each window was calculated, and three data points near the maximum value were retained. The remaining data points were the outliers of this sliding window.

[0024] A sliding window is set for the gamma wave in the third emotional state. The maximum value in each window is calculated, and three data points near the maximum value are retained. The remaining data points are the outliers of this sliding window.

[0025] In addition to the aspects and any possible implementations described above, a further implementation is provided in which, in step S2, an electroencephalogram (EEG) is drawn using locally magnified EEG mapping values, alpha waves, beta waves, and gamma waves are identified, and the emotional state corresponding to the EEG sampling interval is determined.

[0026] On the other hand, the present invention also provides a system for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values, the system being used to implement the above-described method, the system comprising:

[0027] The brainwave acquisition and processing unit is used to acquire brainwave mapping values ​​and separate the alpha, beta, and gamma waves of the brainwaves.

[0028] The emotional state judgment unit determines whether a sampling interval is a first emotional state, a second emotional state, or a third emotional state based on the values ​​of alpha, beta, and gamma waves in each sampling interval and according to a set criterion.

[0029] The abnormal data removal unit collects EEG mapping values ​​according to different emotional states, and removes abnormal data from the collected data for each emotional state.

[0030] In addition to the aspects described above and any possible implementations, an implementation is further provided in which the first emotional state corresponds to a human calm state, the second emotional state corresponds to a human focused state, and the third emotional state corresponds to a human happy state.

[0031] On the other hand, the present invention also provides an application of the above-mentioned method for constructing an emotion recognition dataset based on electroencephalogram mapping values, characterized in that the application includes:

[0032] The method described above is used to process the collected EEG mapping values ​​to obtain an EEG emotion recognition dataset with abnormal data removed.

[0033] Establish a machine learning or deep learning model for EEG emotion recognition, and train the model using the EEG emotion recognition dataset with outlier data removed to obtain the trained model.

[0034] The trained model is used to identify emotions from the brainwaves to be identified.

[0035] The beneficial effects of this invention are as follows: the method of this invention provides a solution for removing outliers in brainwaves of different emotions, and solves the problems of difficulty in constructing brainwave emotion recognition data and the large number of outliers. Attached Figure Description

[0036] Figure 1 The image shown is a waveform diagram of a complete EEG mapping obtained from a test subject.

[0037] Figure 2 The image shows the source. Figure 1 The waveform of the first emotional state is extracted from the waveform diagram.

[0038] Figure 3 The image shows the source. Figure 1 The waveform of the second emotional state is extracted from the waveform diagram.

[0039] Figure 4 The image shows the source. Figure 1 The waveform of the third emotional state is extracted from the waveform diagram.

[0040] Figure 5 As shown Figure 2 Waveform of the first emotional state after removing outliers.

[0041] Figure 6 As shown Figure 3 Waveform of the second emotional state after removing outliers (sliding window parameter 10, step size 3).

[0042] Figure 7 As shown Figure 4Waveform of the third emotional state after removing outliers (sliding window parameter 10, step size 3).

[0043] Figure 8 The diagram shown is a flowchart illustrating a method for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values ​​according to an embodiment of the present invention. Detailed Implementation

[0044] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be noted that the technical features or combinations of technical features described in the following embodiments should not be considered in isolation, but can be combined with each other to achieve better technical effects.

[0045] Brainwaves are the macroscopic manifestations of the electrical activity of neurons in the brain. Based on different frequency ranges, brainwaves can be divided into several main types:

[0046] Delta waves: Frequency range: 0.5-4 Hz

[0047] Characteristics: It appears during deep sleep and is the slowest brainwave.

[0048] Theta waves: Frequency range: 4-8 Hz

[0049] Characteristics: Commonly found in the brains of children and adolescents, it is associated with memory, creativity, mood regulation, and deep relaxation.

[0050] Alpha waves: Frequency range: 8-13 Hz

[0051] Characteristics: Associated with relaxation, meditation, and light sleep states, typically occurring when resting with eyes closed.

[0052] Beta waves: Frequency range: 13-30 Hz

[0053] Characteristics: Associated with alertness, logical thinking, and everyday cognitive activities.

[0054] Gamma waves: Frequency range: above 30 Hz

[0055] Characteristics: It is associated with the synchronized activity of cognitive functions, attention, consciousness, and neural networks.

[0056] σ waves: Frequency range: 12-16 Hz

[0057] Characteristics: Associated with light sleep stages and certain types of sleep disorders.

[0058] μ waves: Frequency range: 7-13 Hz

[0059] Characteristics: It is associated with motor control and sensory inhibition, and usually occurs during fine motor activities.

[0060] Specific waveform changes are associated with specific human emotional states. This invention focuses on the correlation between alpha, beta, and gamma waves and human emotional states. Slow, relatively flat waveforms are associated with calm emotions; beta waves with many peaks and rapidly fluctuating peaks are associated with focused emotions; and gamma waves with the most peaks and extremely rapid fluctuations are associated with happy emotions. Based on the distribution characteristics of different waves, this invention divides the data into EEG data for these three emotions.

[0061] As shown in the figure, an embodiment of the present invention provides a method for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values, comprising:

[0062] S1. Collect EEG mapping values ​​through multiple sampling points;

[0063] S2. Based on the collected EEG mapping values, separate the alpha, beta, and gamma waves of the EEG, and determine whether the sampling interval is the first emotional state, the second emotional state, or the third emotional state according to the values ​​of the alpha, beta, and gamma waves in each sampling interval and the set criteria.

[0064] S3. Collect EEG mapping values ​​according to different emotional states, and remove abnormal data from the collected data for each emotional state to obtain an EEG emotion recognition dataset.

[0065] The specific process is as follows: Figure 8 As shown.

[0066] In one specific embodiment, in step S2, multiple sampling points are arranged in chronological order, with each m1 point forming a sampling interval, for a total of n sampling intervals;

[0067] The criteria for determining a sampling interval as the second emotional state are as follows: if there are m2 or more first peak points in the beta wave within the sampling interval, then the sampling interval is determined to be the second emotional state; the first peak point is a sampling point whose beta wave mapping value is greater than the mean of the beta wave mapping values ​​of all sampling points in the interval by more than three standard deviations.

[0068] The criteria for determining a sampling interval as the third emotional state are as follows: if there are m3 or more second peak points of the gamma wave within the sampling interval, then the sampling interval is determined to be the third emotional state; the second peak point is a sampling point where the gamma wave mapping value is greater than the mean of the gamma wave mapping values ​​of all sampling points in the interval by more than three standard deviations.

[0069] The criteria for determining the sampling interval as the first emotional state are: if the alpha wave, beta wave, and gamma wave are relatively stable within the sampling interval and have fewer than m4 peak points, and m4≤min(m2,m3), then the sampling interval is determined to be the first emotional state.

[0070] n, m1, m2, m3, and m4 are all natural numbers. The ratios of m2 to m1 and m3 to m1 are determined through experiments.

[0071] In one specific embodiment, in step S2, the sampling time interval is 1 second, the total number of sampling points is 360, and every 10 sampling points constitute a sampling interval, for a total of 36 sampling intervals;

[0072] The criterion for determining a sampling interval as a second emotional state is: if the beta wave has three or more first peak points within the sampling interval, then the sampling interval is determined to be a second emotional state.

[0073] The criterion for determining a sampling interval as the third emotional state is: if the gamma wave has 4 or more second peak points within the sampling interval, then the sampling interval is determined to be the third emotional state.

[0074] The criteria for determining the first emotional state in a sampling interval are: if the alpha, beta, and gamma waves are relatively stable within the sampling interval and each has fewer than 3 peak points, then the sampling interval is determined to be the first emotional state.

[0075] In one specific embodiment, the first emotional state is associated with a human calm state, the second emotional state is associated with a human focused state, and the third emotional state is associated with a human happy state.

[0076] In one specific embodiment, in step S1, the original brainwaves of the test subject are collected using an electroencephalogram (EEG) sensor. The original brainwaves are then locally amplified and characterized by brainwave mapping values ​​to avoid the phenomenon that the original brainwaves have indistinct features due to their small values.

[0077] In one specific embodiment, in step S3, abnormal mapping values ​​in the first emotional state data are automatically removed using box plot rules, and abnormal mapping values ​​in the second and third emotional state data are automatically removed using sliding window local extremum filtering.

[0078] In one specific embodiment, the method for removing abnormal mapping values ​​is as follows:

[0079] All EEG mapping values ​​for the first emotional state are arranged from smallest to largest to obtain the first and third quartiles of each type of alpha, beta, and gamma wave. The IQR (Interquartile Range) is obtained by subtracting the first quartile from the third quartile. The lower bound is the IQR value located at 1.5 times the first quartile, and the upper bound is the IQR value located at 1.5 times the third quartile. Data outside the upper and lower bounds are considered abnormal EEG waves in the first emotional state data.

[0080] Calculate IQR (Interquartile Range):

[0081] IQR = Q3 - Q1 (1)

[0082] Where Q3 is the third quantile and Q1 is the first quantile.

[0083] Calculate the lower bound N and the upper bound M:

[0084] N = Q1 - 1.5IQR (2)

[0085] M = Q³ - 1.5IQR (3)

[0086] A sliding window was set for the β wave in the second emotional state. The maximum value in each window was calculated, and two data points to the left and right of the maximum value were retained. The remaining data points were the outliers of this sliding window.

[0087] A sliding window was set for the gamma wave in the third emotional state. The maximum value in each window was calculated, and two data points to the left and right of the maximum value were retained. The remaining data points were the outliers of this sliding window.

[0088] In one specific embodiment, an electroencephalogram (EEG) is drawn using locally magnified EEG mapping values ​​to identify alpha, beta, and gamma waves, and to determine the emotional state corresponding to the EEG sampling interval.

[0089] This invention provides a system for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values. The system is used to implement the above-described method and includes:

[0090] The brainwave acquisition and processing unit is used to acquire brainwave mapping values ​​and separate the alpha, beta, and gamma waves of the brainwaves.

[0091] The emotional state judgment unit determines whether a sampling interval is a first emotional state, a second emotional state, or a third emotional state based on the values ​​of alpha, beta, and gamma waves in each sampling interval and according to a set criterion.

[0092] The abnormal data removal unit collects EEG mapping values ​​according to different emotional states, and removes abnormal data from the collected data for each emotional state.

[0093] In one specific embodiment, an electroencephalogram (EEG) sensor is used to collect brainwaves from multiple test subjects. The sensor is equipped with a local amplification module, which can directly obtain the brainwave mapping value.

[0094] To efficiently collect brainwaves for various emotions, test subjects watched video clips that could evoke the first emotional state (calm state), the second emotional state (focused state), and the third emotional state (happy state) during the collection process.

[0095] An application of the above-mentioned method for constructing emotion recognition datasets based on electroencephalogram (EEG) mapping values ​​is provided in this embodiment of the invention. The application includes:

[0096] The method described above is used to process the collected EEG mapping values ​​to obtain an EEG emotion recognition dataset with abnormal data removed.

[0097] Establish a machine learning or deep learning model for EEG emotion recognition, and train the model using the EEG emotion recognition dataset with outlier data removed to obtain the trained model.

[0098] The trained model is used to identify emotions from the brainwaves to be identified.

[0099] Example

[0100] Figure 1 A waveform diagram of the entire EEG mapping value collected for a test subject.

[0101] The data sampling frequency is once per second, and the entire data segment consists of 360 sampling points. The data is divided into sampling intervals of 10 sampling points each. Figure 2 , Figure 3 , Figure 4 The waveforms are for the three emotional states: the first emotional state (calm), the second emotional state (focused), and the third emotional state (happy).

[0102] The "box plot rule" is used to automatically remove outlier values ​​from the first emotional state (calm) data. Figure 5 This is a waveform of the first emotional state after removing outliers.

[0103] The "sliding window local extremum filtering method" is used to automatically remove abnormal mapping values ​​in the data of the second emotional state (focus) and the third emotional state (happiness). The sliding window parameter is 10 and the step size is 3. Figure 6 , Figure 7These are waveforms of the second and third emotional states after outliers have been removed.

[0104] After the above process of removing outlier data, an emotion recognition dataset with 54 data points, each with 120 sampling points, and a total of 6480 sampling points was obtained.

[0105] To verify the objectivity and effectiveness of the method of this invention, the applicant conducted multiple comparative experiments. During the acquisition of brainwaves, artificial interference (e.g., electrical noise and bioelectrical noise) was added. The interfered brainwaves were collected, and the brainwave data processed by the method of this invention significantly removed the abnormal data caused by electrical and bioelectrical noise. Simultaneously, the mutual interference between different emotional states was also effectively removed. This demonstrates the high efficiency of this invention in removing abnormal data.

[0106] The foregoing description illustrates and describes several preferred embodiments of this application. However, as previously stated, it should be understood that this application is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the application concept described herein through the foregoing teachings or techniques or knowledge in related fields. Any modifications and variations made by those skilled in the art that do not depart from the spirit and scope of this application should be within the protection scope of the appended claims.

Claims

1. A method for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values, characterized in that, The method includes: S1. Collect EEG mapping values ​​through multiple sampling points; S2. Based on the collected EEG mapping values, separate the alpha, beta, and gamma waves of the EEG, and determine whether the sampling interval is the first emotional state, the second emotional state, or the third emotional state according to the values ​​of the alpha, beta, and gamma waves in each sampling interval and the set criteria. S3. Collect brainwave mapping values ​​according to different emotional states, and remove abnormal data from the collected data for each emotional state to obtain a brainwave emotion recognition dataset. In step S2, multiple sampling points are arranged in chronological order, with each sampling interval consisting of m1 points, for a total of n sampling intervals; The criteria for determining a sampling interval as the second emotional state are as follows: if there are m2 or more first peak points in the beta wave within the sampling interval, then the sampling interval is determined to be the second emotional state; the first peak point is a sampling point whose beta wave mapping value is greater than the mean of the beta wave mapping values ​​of all sampling points in the interval by more than three standard deviations. The criteria for determining a sampling interval as the third emotional state are as follows: if there are m3 or more second peak points of the gamma wave within the sampling interval, then the sampling interval is determined to be the third emotional state; the second peak point is a sampling point where the gamma wave mapping value is greater than the mean of the gamma wave mapping values ​​of all sampling points in the interval by more than three standard deviations. The criteria for determining the sampling interval as the first emotional state are: if the alpha wave, beta wave, and gamma wave are relatively stable within the sampling interval and have fewer than m4 peak points, and m4≤min(m2,m3), then the sampling interval is determined to be the first emotional state. n, m1, m2, m3, and m4 are all natural numbers; In step S3, box plot rules are used to automatically remove abnormal mapping values ​​from the first emotional state data, and sliding window local extremum filtering method is used to automatically remove abnormal mapping values ​​from the second and third emotional state data. The specific method for removing abnormal mapping values ​​is as follows: All EEG mapping values ​​for the first emotional state are arranged from smallest to largest to obtain the first and third quartiles of each type of alpha, beta, and gamma wave. The IQR is obtained by subtracting the first quartile from the third quartile. The lower bound is the IQR value located at the first quartile minus 1.5 times, and the upper bound is the IQR value located at the third quartile plus 1.5 times. Data outside the upper and lower bounds are considered abnormal EEG waves in the first emotional state data. A sliding window was set for the β wave in the second emotional state. The maximum value in each window was calculated, and two data points to the left and right of the maximum value were retained. The remaining data points were the outliers of this sliding window. A sliding window is set for the gamma wave in the third emotional state. The maximum value in each window is calculated, and two data points to the left and right of the maximum value are retained. The remaining data points are the outliers of this sliding window.

2. The method for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values ​​as described in claim 1, characterized in that, In step S2, the sampling time interval is 1 second, the total number of sampling points is 360, and every 10 sampling points constitute a sampling interval, for a total of 36 sampling intervals; The criterion for determining a sampling interval as a second emotional state is: if the beta wave has three or more first peak points within the sampling interval, then the sampling interval is determined to be a second emotional state. The criterion for determining a sampling interval as the third emotional state is: if the gamma wave has 4 or more second peak points within the sampling interval, then the sampling interval is determined to be the third emotional state. The criteria for determining the first emotional state in a sampling interval are: if the alpha, beta, and gamma waves are relatively stable within the sampling interval and each has fewer than 3 peak points, then the sampling interval is determined to be the first emotional state.

3. The method for constructing an emotion recognition dataset based on electroencephalogram mapping values ​​as described in claim 1, characterized in that, In step S1, the raw brain waves of the test subject are collected using an electroencephalogram (EEG) sensor, and the raw brain waves are locally amplified and represented by brain wave mapping values.

4. The method for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values ​​as described in claim 1, characterized in that, In step S2, the EEG is drawn using the locally magnified EEG mapping values, the alpha, beta, and gamma waves are identified, and the emotional state corresponding to the EEG sampling interval is determined.

5. A system for constructing an emotion recognition dataset based on electroencephalogram (EEG) mapping values, characterized in that, The system is used to implement the method as described in any one of claims 1-4, the system comprising: The brainwave acquisition and processing unit is used to acquire brainwave mapping values ​​and separate the alpha, beta, and gamma waves of the brainwaves. The emotional state judgment unit determines whether a sampling interval is a first emotional state, a second emotional state, or a third emotional state based on the values ​​of alpha, beta, and gamma waves in each sampling interval and according to a set criterion. The abnormal data removal unit collects EEG mapping values ​​according to different emotional states, and removes abnormal data from the collected data for each emotional state.

6. The emotion recognition dataset construction system based on EEG mapping values ​​as described in claim 5, characterized in that, The first emotional state corresponds to a calm state in humans, the second emotional state corresponds to a focused state in humans, and the third emotional state corresponds to a happy state in humans.

7. An application of the method for constructing an emotion recognition dataset based on electroencephalogram mapping values ​​as described in any one of claims 1-4, characterized in that, The applications include: The method described above is used to process the collected EEG mapping values ​​to obtain an EEG emotion recognition dataset with abnormal data removed. Establish a machine learning or deep learning model for EEG emotion recognition, and train the model using the EEG emotion recognition dataset with outlier data removed to obtain the trained model. The trained model is used to identify emotions from the brainwaves to be identified.

Images