Electroencephalogram data processing method, device, system, computer device and storage medium

Abstract

The application discloses an electroencephalogram data processing method, device and system, computer equipment and a storage medium. The method comprises the following steps: acquiring electroencephalogram data of a target object; performing a preprocessing operation on the electroencephalogram data to obtain target electroencephalogram data; determining a current emotional state of the target object according to the target electroencephalogram data, wherein the current emotional state is state data associated with the emotion of the target object; and performing corresponding display of the current emotional state of the target object in a display interface. The method can improve the emotion prediction accuracy.

Description

Technical Field

[0001] This invention relates to the field of electroencephalography (EEG) technology, and in particular to an EEG data processing method, apparatus, system, computer equipment, and storage medium. Background Technology

[0002] As living standards improve, people have higher expectations for all aspects of life. Businesses need to evaluate the current quality of their services in order to make corresponding adjustments and provide users with better services.

[0003] For example, in a teaching scenario, teachers need to evaluate the current teaching quality to adjust teaching methods and content based on the evaluation results. Currently, the evaluation is based on student feedback or exam scores. However, student feedback and exam scores are subjective, leading to low accuracy in the evaluation and preventing real-time assessment. Similarly, in a gaming scenario, game developers need to evaluate user experience. Current methods rely on user playtime and feedback. However, playtime and feedback only reflect user feelings to a certain extent and remain subjective, resulting in low accuracy and preventing real-time evaluation.

[0004] Therefore, the evaluation methods using user feedback or business-related data in related technologies have low accuracy and poor real-time performance. Summary of the Invention

[0005] One technical problem that this invention aims to solve is that evaluation methods using user feedback or business-related data often result in low accuracy and poor real-time performance of the evaluation results.

[0006] In a first aspect, the present invention provides a method for processing electroencephalogram (EEG) data, the method comprising:

[0007] Acquire the electroencephalogram (EEG) data of the target subject;

[0008] The EEG data is preprocessed to obtain the target EEG data;

[0009] Based on the target EEG data, the current emotional state of the target object is determined, and the current emotional state is state data associated with the target object's emotions;

[0010] The current emotional state of the target object is displayed in the display interface.

[0011] In one embodiment, the current emotional state includes at least one of the following: emotion category, emotional change trajectory, focus level, percentage of each emotion category during the monitoring period, and EEG components of the EEG data.

[0012] In one embodiment, the current emotional state includes an emotion category, and determining the current emotional state of the target object based on the target EEG data includes:

[0013] Feature extraction is performed on the target EEG data to obtain a first EEG feature and a second EEG feature, wherein the first EEG feature is the time domain feature of the target EEG data and the second EEG feature is the frequency domain feature of the target EEG data;

[0014] An emotion prediction model is used to perform emotion prediction processing on the first EEG feature and the second EEG feature to obtain the emotion category of the target object at the current moment.

[0015] In one embodiment, prior to performing feature extraction on the target EEG data, the method includes:

[0016] For any channel of the target EEG data, determine the similarity between the EEG data corresponding to that channel and the electrooculogram template;

[0017] Based on the similarity between the EEG data corresponding to each channel and the EEG template, the target EEG data is subjected to EEG artifact removal processing to obtain artifact-removed target EEG data.

[0018] In one embodiment, the method further includes:

[0019] Multiple sample EEG data were collected, wherein the sample EEG data were collected when the sampled subject blinked;

[0020] The EEG data of the multiple samples are fused to obtain the EEG data of the target sample.

[0021] Component analysis is performed on the EEG data of the target sample to obtain the EEG components corresponding to the EEG data of the target sample;

[0022] The electrooculogram (EEG) components were extracted to obtain the electrooculogram template.

[0023] In one embodiment, the current emotional state further includes focus, and determining the current emotional state of the target object based on the target EEG data further includes:

[0024] The data in each channel of the target EEG data is divided into multiple windows;

[0025] A blink window is determined from the plurality of windows, wherein the standard deviation of the EEG data within the blink window is greater than a standard deviation threshold;

[0026] The focus level of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

[0027] In one embodiment, determining the target object's level of focus at the current moment based on the total number of blink windows and the total number of channels in the target EEG data includes:

[0028] The initial level of focus of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

[0029] The historical focus level and initial focus level of the target object within a first preset time period are smoothed to obtain the focus level of the target object at the current moment.

[0030] In one embodiment, the current emotional state further includes an emotional change trajectory, and determining the current emotional state of the target object based on the target EEG data further includes:

[0031] Based on the target object's current emotion category and the target object's current level of focus, determine the target object's coordinate information in the flow coordinate system. The horizontal axis of the flow coordinate system is used to represent the target object's level of focus, and the vertical axis of the flow coordinate system is used to represent the target object's rate of emotion change.

[0032] In the flow trajectory area of ​​the display interface, the flow marker is moved according to the coordinate information, and the emotional change trajectory corresponding to the target object is obtained according to the movement trajectory of the flow marker.

[0033] The flow trajectory area includes multiple display areas, each corresponding to a different emotion category.

[0034] In one embodiment, the flow trajectory region is divided into four display areas by the four quadrants of the flow coordinate system. The four display areas include: the first quadrant of the flow coordinate system corresponds to the first emotion and the first display area; the second quadrant of the flow coordinate system corresponds to the second emotion and the second display area; the third quadrant of the flow coordinate system corresponds to the third emotion and the third display area; and the fourth quadrant of the flow coordinate system corresponds to the fourth emotion and the fourth display area.

[0035] In one embodiment, determining the coordinate information of the target object in the flow coordinate system based on the target object's current emotion category and the target object's current focus level includes:

[0036] Based on the target object's current emotion category and the target object's current level of focus, determine the target object's horizontal coordinate in the flow coordinate system;

[0037] Based on the difference between two emotion categories of the target object at adjacent times within a second preset time period, the vertical coordinate of the target object in the flow coordinate system is determined.

[0038] In one embodiment, an energy marker is displayed at the location where the flow marker in the emotion change trajectory has stayed, and the size and / or brightness of the energy marker is negatively correlated with the display duration of the energy marker.

[0039] Secondly, the present invention also provides an electroencephalogram (EEG) data processing system, the system comprising an EEG data acquisition device, a data processing device, and a display device; wherein,

[0040] The EEG data acquisition is used to collect the EEG data of the target object and send the EEG data of the target object to the data processing device;

[0041] The data processing device is used to preprocess the EEG data of the target object, and after obtaining the target EEG data, determine the current emotional state of the target object based on the target EEG data.

[0042] The display device is configured to, in response to a viewing operation on the target object, display the current emotional state of the target object in a display interface.

[0043] Thirdly, the present invention also provides an electroencephalogram (EEG) data processing device, the device comprising:

[0044] The acquisition module is used to acquire the EEG data of the target object;

[0045] The preprocessing module is used to perform preprocessing operations on the EEG data to obtain the target EEG data;

[0046] The first determining module is used to determine the current emotional state of the target object based on the target EEG data, wherein the current emotional state is state data associated with the emotion of the target object;

[0047] The display module is used to display the current emotional state of the target object in the display interface.

[0048] In one embodiment, the current emotional state includes at least one of the following: emotion category, emotional change trajectory, focus level, percentage of each emotion category during the monitoring period, and EEG components of the EEG data.

[0049] In one embodiment, the current emotional state includes an emotional category, and the first determining module is further configured to:

[0050] Feature extraction is performed on the target EEG data to obtain a first EEG feature and a second EEG feature, wherein the first EEG feature is the time domain feature of the target EEG data and the second EEG feature is the frequency domain feature of the target EEG data;

[0051] An emotion prediction model is used to perform emotion prediction processing on the first EEG feature and the second EEG feature to obtain the emotion category of the target object at the current moment.

[0052] In one embodiment, the device further includes:

[0053] The second determining module is used to determine the similarity between the EEG data corresponding to any channel of the target EEG data and the electrooculogram template.

[0054] The artifact removal module is used to perform EEG artifact removal processing on the target EEG data based on the similarity between the EEG data corresponding to each channel and the EEG template, so as to obtain artifact-removed target EEG data.

[0055] In one embodiment, the device further includes:

[0056] The acquisition module is used to acquire multiple sample EEG data, wherein the sample EEG data is acquired when the sampled object blinks;

[0057] The fusion module is used to fuse the multiple sample EEG data to obtain the target sample EEG data;

[0058] The analysis module is used to perform component analysis on the target sample EEG data to obtain the EEG components corresponding to the target sample EEG data;

[0059] An extraction module is used to extract electrooculography (EOG) components from the electroencephalogram (EEG) components to obtain the EOG template.

[0060] In one embodiment, the current emotional state further includes focus, and the first determining module is further configured to:

[0061] The data in each channel of the target EEG data is divided into multiple windows;

[0062] A blink window is determined from the plurality of windows, wherein the standard deviation of the EEG data within the blink window is greater than a standard deviation threshold;

[0063] The focus level of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

[0064] In one embodiment, the first determining module is further configured to:

[0065] The initial level of focus of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

[0066] The historical focus level and initial focus level of the target object within a first preset time period are smoothed to obtain the focus level of the target object at the current moment.

[0067] In one embodiment, the current emotional state further includes an emotional change trajectory, and the first determining module is further configured to:

[0068] Based on the target object's current emotion category and the target object's current level of focus, determine the target object's coordinate information in the flow coordinate system. The horizontal axis of the flow coordinate system is used to represent the target object's level of focus, and the vertical axis of the flow coordinate system is used to represent the target object's rate of emotion change.

[0069] In the flow trajectory area of ​​the display interface, the flow marker is moved according to the coordinate information, and the emotional change trajectory corresponding to the target object is obtained according to the movement trajectory of the flow marker.

[0070] The flow trajectory area includes multiple display areas, each corresponding to a different emotion category.

[0071] In one embodiment, the flow trajectory region is divided into four display areas by the four quadrants of the flow coordinate system. The four display areas include: the first quadrant of the flow coordinate system corresponds to the first emotion and the first display area; the second quadrant of the flow coordinate system corresponds to the second emotion and the second display area; the third quadrant of the flow coordinate system corresponds to the third emotion and the third display area; and the fourth quadrant of the flow coordinate system corresponds to the fourth emotion and the fourth display area.

[0072] In one embodiment, the first determining module is further configured to:

[0073] Based on the target object's current emotion category and the target object's current level of focus, determine the target object's horizontal coordinate in the flow coordinate system;

[0074] Based on the difference between two emotion categories of the target object at adjacent times within a second preset time period, the vertical coordinate of the target object in the flow coordinate system is determined.

[0075] In one embodiment, an energy marker is displayed at the location where the flow marker in the emotion change trajectory has stayed, and the size and / or brightness of the energy marker is negatively correlated with the display duration of the energy marker.

[0076] Fourthly, the present invention also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the electroencephalogram (EEG) data processing method described above.

[0077] Fifthly, the present invention also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the electroencephalogram (EEG) data processing method described above.

[0078] Sixthly, the present invention also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the electroencephalogram (EEG) data processing method described above.

[0079] The aforementioned EEG data processing method, apparatus, system, computer equipment, and storage medium can preprocess the EEG data of the target object after acquisition to obtain target EEG data, determine the target object's current emotional state based on the target EEG data, and display the target object's current emotional state in a display interface. The current emotional state is state data associated with the target object's emotion. Based on the EEG data processing method, apparatus, system, computer equipment, and storage medium provided in this disclosure, the current emotional state of the target object can be acquired and displayed in real time using the target object's EEG data. This allows users to obtain the target object's emotional state in real time and then evaluate business operations based on the target object's emotional state. Since emotional states can more realistically represent the user's feelings and are not affected by the target object's subjective consciousness, this not only improves the accuracy of business evaluation but also enhances the real-time nature of the business evaluation.

[0080] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0081] The accompanying drawings, which form part of this specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0082] The invention will be more clearly understood with reference to the accompanying drawings and the following detailed description, wherein:

[0083] Figure 1a This is an application environment diagram of an EEG data processing method in one embodiment;

[0084] Figure 1b This is an application environment diagram of an EEG data processing method in one embodiment;

[0085] Figure 2 This is a flowchart illustrating an EEG data processing method in one embodiment;

[0086] Figure 3 This is a schematic diagram of an EEG data processing method in one embodiment;

[0087] Figure 4 This is a flowchart illustrating step 206 in one embodiment;

[0088] Figure 5 This is a schematic diagram illustrating the interface of an EEG data processing method in one embodiment;

[0089] Figure 6 This is a flowchart illustrating an EEG data processing method in one embodiment;

[0090] Figure 7 This is a flowchart illustrating an EEG data processing method in one embodiment;

[0091] Figure 8 This is a flowchart illustrating an EEG data processing method in one embodiment;

[0092] Figure 9 This is a flowchart illustrating an EEG data processing method in one embodiment;

[0093] Figure 10 This is a flowchart illustrating step 206 in one embodiment;

[0094] Figure 11 This is a flowchart illustrating an EEG data processing method in one embodiment;

[0095] Figure 12 This is a flowchart illustrating step 1006 in one embodiment;

[0096] Figure 13 This is a flowchart illustrating step 206 in one embodiment;

[0097] Figure 14 This is a flowchart illustrating step 1302 in one embodiment;

[0098] Figure 15 This is a schematic diagram of an EEG data processing method in one embodiment;

[0099] Figure 16 This is a structural block diagram of an EEG data processing device in one embodiment;

[0100] Figure 17 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0101] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0102] At the same time, it should be understood that, for ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn according to actual scale.

[0103] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0104] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, such techniques, methods, and equipment should be considered part of the specification.

[0105] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0106] Embodiments of this invention can be applied to computer systems / servers that can operate with a wide range of other general-purpose or special-purpose computing system environments or configurations. Examples of well-known computing systems, environments, and / or configurations suitable for use with computer systems / servers include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the above systems, etc.

[0107] Computer systems / servers can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. Typically, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in distributed cloud computing environments, where tasks are performed by remote processing devices linked through a communication network. In distributed cloud computing environments, program modules can reside on local or remote computing system storage media, including storage devices.

[0108] The EEG data processing method provided in this embodiment of the invention can be applied to, for example... Figure 1a In the application environment shown, the EEG data acquisition device 102 communicates with the data processing device 104 via a network. The data storage system can store the data that the data processing device 104 needs to process. The data storage system can be integrated into the data processing device 104 or placed in the cloud or on another network server. The EEG data acquisition device 102 can acquire the target object's EEG data in real time and send the target object's EEG data to the data processing device 104. The data processing device 104 can preprocess the target object's EEG data to obtain target EEG data, and based on the target EEG data, determine the target object's current emotional state, including at least one of the following: emotion category, emotional change trajectory, focus level, proportion of each emotion category during the monitoring period, and EEG components of the EEG data. The display device can display the target object's current emotional state in a display interface. For example, this embodiment of the invention can be applied to... Figure 1b The scenario shown is used for teaching quality assessment.

[0109] The data processing device 104 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. Portable wearable devices can include smartwatches, smart bracelets, head-mounted devices, etc., and can also be implemented using a standalone server or a server cluster composed of multiple servers.

[0110] In one embodiment, such as Figure 2 As shown, a method for processing electroencephalogram (EEG) data is provided. Taking the application of this method to the data processing device 104 in Figure 1 as an example, the method includes the following steps:

[0111] Step 202: Obtain the EEG data of the target subject.

[0112] In this embodiment of the disclosure, the data processing device can be communicatively connected to the EEG data acquisition device to acquire the EEG data of the target object. The EEG data acquisition device may include a brain-computer interface device or an AR (Augmented Reality) device. If the EEG data acquisition device is an AR device, a forehead EEG monitoring module can be integrated into the frame of the AR device's glasses to acquire the target object's EEG data.

[0113] The EEG data acquisition device and the data processing device can communicate using wireless communication methods such as networks and Bluetooth, or they can communicate using wired communication methods such as data cables. This disclosure does not specifically limit the communication methods.

[0114] For example, the EEG data acquisition device can acquire the EEG data of the target object in real time, and the specific acquisition frequency can be set by those skilled in the art according to their needs. The data processing device receives the target object's EEG data from the EEG data acquisition device in real time and can acquire the EEG data within the sliding window at the current moment. The window duration of the sliding window is a preset duration (the specific value is not specifically limited by those skilled in the art). For example, if the window duration of the sliding window is 5 seconds, then if the current moment is the 10th second, the target object's EEG data from the 6th to the 10th second can be acquired.

[0115] Step 204: Perform preprocessing operations on the EEG data to obtain the target EEG data.

[0116] In this embodiment of the disclosure, after acquiring the EEG data of the target object, preprocessing operations can be performed on the acquired EEG data to remove interference data and redundant information, thereby obtaining the target EEG data. The preprocessing operations for the EEG data may include high-pass filtering, low-pass filtering, etc., as exemplarily referred to... Figure 3 As shown, EEG data can be high-pass filtered through a 45-55Hz band-stop filter to remove 50Hz power frequency interference, and then low-pass filtered through a 0.5-50Hz band-pass filter to extract the effective components from the EEG data. This embodiment does not specifically limit the preprocessing operation.

[0117] Step 206: Determine the current emotional state of the target based on the target EEG data. The current emotional state is the state data associated with the target's emotions.

[0118] In this embodiment of the disclosure, after obtaining the target EEG data of the target object, the current emotional state of the target object can be determined based on the target EEG data. For example, the target EEG data of the target object can be analyzed or predicted to obtain the current emotional state of the target object. The current emotional state may include state data related to the target object's emotions, such as data that can characterize the user's current emotions, current level of focus, historical emotions, and emotional changes.

[0119] Step 208: Display the current emotional state of the target object in the display interface.

[0120] In this embodiment of the disclosure, after determining the current emotional state of the target object, the current emotional state of the target object can be displayed through the display interface of the display device, so that the user can obtain the current emotional state of the target object through the information displayed on the display interface.

[0121] The display device may be integrated into the data processing device, or the display device may be an external device of the data processing device. This disclosure does not specifically limit this aspect.

[0122] The EEG data processing method provided in this disclosure can preprocess the EEG data of a target object after acquisition to obtain target EEG data, determine the target object's current emotional state based on the target EEG data, and display the target object's current emotional state in a display interface. The current emotional state is state data associated with the target object's emotions. Based on the EEG data processing method provided in this disclosure, the current emotional state of a target object can be obtained and displayed in real time using the target object's EEG data. This allows users to obtain the target object's emotional state in real time and then evaluate business operations based on the target object's emotional state. Since emotional states can more realistically represent the user's feelings and are not affected by the target object's subjective consciousness, this not only improves the accuracy of business evaluation but also enhances the real-time nature of the business evaluation.

[0123] In one embodiment, the current emotional state includes at least one of the following: emotion category, emotional change trajectory, focus level, percentage of each emotion category during the monitoring period, and EEG components of the EEG data.

[0124] In this embodiment of the disclosure, the current emotional state may include at least one of the following: emotion category, emotional change trajectory, focus level, proportion of each emotion category during the monitoring period, and EEG components of the EEG data. That is, the display interface may display at least one of the following: the target object's emotion category at the current moment, emotional change trajectory, focus level, proportion of each emotion category during the monitoring period, and EEG components of the EEG data.

[0125] For example, this disclosure does not specifically limit the categories of emotions. Those skilled in the art can categorize emotions according to their needs. For instance, emotion categories may include feelings of ease, excitement, anxiety, boredom, etc., or they may be further categorized into higher dimensions as needed. Emotional change trajectories can be used to characterize a user's emotional changes within a monitoring period and can be displayed as a trajectory graph or an emotion list, etc. This disclosure does not specifically limit the display format of the emotional change trajectory.

[0126] Attention level can be used to characterize the degree of focus of a user at a given moment. This disclosure does not specifically limit the form of attention level. Any method that can characterize a user's attention level is applicable to this disclosure. For example, a user's attention level can be represented as a value from 1 to 100, with a larger value indicating a higher degree of focus; or, a user's attention level can be represented as a corresponding level, with a higher level indicating a higher degree of focus.

[0127] The system can count the number of times each emotion category appears in the target object during the monitoring period in real time, and also count the percentage of each emotion category in the total number of emotion categories in real time, thereby obtaining the percentage of each emotion category during the monitoring period. This embodiment does not specify the way the percentage of each emotion category is displayed during the monitoring period. For example, it can be displayed by displaying percentage values, or by displaying statistical charts, etc.

[0128] By performing component analysis on the EEG data of the target object, the corresponding EEG components can be obtained. The display interface can show a three-dimensional model of the human brain. In this embodiment of the disclosure, the EEG data can be displayed in the three-dimensional model of the human brain according to the EEG components corresponding to the EEG data. For example, according to the EEG components corresponding to the EEG data, the corresponding EEG signal markers can be displayed at the locations corresponding to the EEG components in the three-dimensional model of the human brain.

[0129] Based on the EEG data processing method provided in this disclosure, information such as the target object's emotion category, emotion change trajectory, focus level, proportion of each emotion category during the monitoring period, and EEG components of the EEG data can be obtained in real time from the target object's EEG data and displayed accordingly. This allows users to obtain the target object's current emotional state in real time, and then evaluate the business in real time based on the target object's current emotional state, which can improve the accuracy of business evaluation and achieve real-time business evaluation.

[0130] In one embodiment, the current emotional state includes an emotional category, as referenced. Figure 4 As shown, in step 206, determining the current emotional state of the target subject based on the target EEG data may include:

[0131] Step 402: Perform feature extraction on the target EEG data to obtain a first EEG feature and a second EEG feature, wherein the first EEG feature is the time domain feature of the target EEG data and the second EEG feature is the frequency domain feature of the target EEG data.

[0132] Step 404: Use an emotion prediction model to perform emotion prediction processing on the first and second EEG features to obtain the emotion category of the target object at the current moment.

[0133] In this embodiment of the disclosure, feature extraction can be performed on the target EEG data to obtain a first EEG feature in the time domain and a second EEG feature in the frequency domain. The first EEG feature may include Hjorth features (including features related to activity, mobility, and complexity), and the second EEG feature may include the differential entropy features of the EEG data rhythms (Delta, Theta, Alpha, Beta, Gamma).

[0134] For example, the Hjorth features (activity, mobility, and complexity) of each channel of the target EEG data and the differential entropy features (Delta, Theta, Alpha, Beta, Gamma) of each channel can be extracted separately. As another example, refer to... Figure 3 As shown in the embodiments of this disclosure, the target EEG data can be further divided into a preset number (a pre-set value, which is not specifically limited in this embodiment, for example, it can be 5) of main frequency bands, and feature extraction can be performed on each frequency band, which can further improve the feature extraction accuracy. The embodiments of this disclosure do not specifically limit the specific process of feature extraction.

[0135] The extraction process of the first EEG feature can refer to formulas (i) to (iii) below. The extraction process of the second EEG feature can refer to formula (iv) below.

[0136] Activity = var(y(t)) (Formula 1)

[0137]

[0138]

[0139]

[0140] Wherein, Activity represents the activity feature in the Hjorth feature, Mobility represents the mobility feature in the Hjorth feature, Complexity represents the complexity feature in the Hjorth feature, DE represents the differential entropy feature, t represents the current time, y(t) represents the EEG data acquired at the current time, [a,b] represents the frequency interval corresponding to the frequency band, and p(x) represents the power spectral density corresponding to the frequency interval [a,b].

[0141] After obtaining the Hjorth features (first EEG feature) and differential entropy features (second EEG feature) extracted from all channels of the target EEG data, the Hjorth features and differential entropy features can be concatenated into a one-dimensional vector to obtain the feature vector corresponding to the target EEG data.

[0142] Reference Figure 3 As shown, the feature vector corresponding to the target EEG data can be input into the emotion prediction model for emotion prediction processing to obtain the emotion category of the target object at the current moment. The emotion prediction model can be a pre-trained model for emotion classification, and may include random forest classifiers, knowledge vector machines, logistic regression networks, convolutional regression neural networks, recurrent regression neural networks, etc. This disclosure does not specifically limit the emotion prediction model, and the training process of the emotion prediction model will not be described in detail. Any training method capable of training the above-mentioned emotion prediction model is applicable to this disclosure.

[0143] After obtaining the target's current emotional category, the corresponding emotional category can be displayed on the interface, allowing users to know the target's current emotional category in real time and then evaluate the business accordingly.

[0144] For example, refer to Figure 5 As shown, the display interface includes an emotion display area that shows the emotion categories. This area can display the target object's current emotion category, including the text corresponding to the emotion category. Different emotion categories in the display interface have different display icons. The display icons corresponding to the target object's current emotion category can also be highlighted, animated, or have their colors changed.

[0145] It should be noted that the display method of emotion categories in the above example is only one example of the display method of emotion categories in the embodiments of this disclosure. In fact, the embodiments of this disclosure do not specifically limit the display method of emotion categories in the display interface. Any display method that can represent the emotion category of the target object at the current moment is applicable to the embodiments of this disclosure.

[0146] Based on the EEG data processing method provided in this disclosure, the first EEG feature in the time domain and the second EEG feature in the frequency domain of the target EEG data can be extracted to predict the emotion category of the target object at the current moment. By using features in both the time domain and the frequency domain for prediction, the accuracy of emotion category prediction can be improved.

[0147] In one embodiment, reference Figure 6 As shown, prior to step 402, the above method may further include:

[0148] Step 602: For any channel of the target EEG data, determine the similarity between the EEG data corresponding to the channel and the EEG template;

[0149] Step 604: Based on the similarity between the EEG data corresponding to each channel and the EEG template, perform EEG artifact removal processing on the target EEG data to obtain artifact-removed target EEG data.

[0150] In this embodiment of the disclosure, reference is made to Figure 7 As shown, before feature extraction from the target EEG data, artifact removal processing of the target EEG data can be performed to remove the electrooculogram (EOG) data from the target EEG data, thereby reducing the interference of the EOG data on emotion prediction and obtaining artifact-removed target EEG data. Then, feature extraction is performed on the artifact-removed target EEG data, and the extracted features are processed by an emotion prediction model to obtain the emotion category of the target object at the current moment.

[0151] For example, for any channel of the target EEG data, the similarity between the EEG data corresponding to that channel and the electrooculogram (EOG) template can be determined. The EOG template can be a pre-constructed template that can be used to describe the EOG components. The similarity between the EEG data corresponding to each channel and the EOG template is determined. The higher the similarity, the greater the probability that the EEG data of that channel is an EOG component. Therefore, the EEG data of that channel can be removed to achieve EOG artifact removal processing and obtain the artifact-removed target EEG data.

[0152] Taking a target EEG data set as three channels with a sampling frequency of 250 Hz and a sliding window duration of 5 seconds as an example, the data dimension of the target EEG data can be represented as 3×1250, and the EEG template can be represented as a 1×1250 row vector. For any channel of EEG data, its similarity to the EEG template can be determined. If the similarity is greater than a preset similarity threshold (a pre-set value; this embodiment does not specify a specific value, for example, the similarity threshold can be set to 0.8), the EEG data of that channel can be removed, and the remaining channel's EEG data becomes the artifact-free target EEG data.

[0153] Based on the EEG data processing method provided in this disclosure, the EEG data of the target can be processed by EEG template to remove artifacts, thereby avoiding the influence of EEG data on emotion prediction and improving the prediction accuracy of the emotion category of the target object.

[0154] In one embodiment, reference Figure 8 As shown, the above method may further include:

[0155] Step 802: Collect EEG data from multiple samples. The sample EEG data are the EEG data collected when the sampled subjects blink.

[0156] Step 804: The EEG data of multiple samples are fused to obtain the EEG data of the target sample;

[0157] Step 806: Perform component analysis on the EEG data of the target sample to obtain the EEG components corresponding to the EEG data of the target sample.

[0158] Step 808: Extract electrooculography (EOG) components from the electroencephalogram (EEG) components to obtain an EOG template.

[0159] In this embodiment of the disclosure, multiple sample EEG data can be pre-collected. For example, EEG data of multiple sampling subjects can be collected while blinking, such as when the sampling subjects blink continuously for 5-10 seconds, and the EEG data of the sampling subjects is collected as sample EEG data.

[0160] Multiple sample EEG data can be preprocessed and then further fused to obtain a target sample EEG data. This disclosure does not specifically limit the fusion method. For example, multiple sample EEG data can be segmented into 5-second sample segments, and all sample segments can be superimposed and averaged to obtain the target sample EEG data.

[0161] After obtaining the EEG data of the target sample, Eeglab can be used to perform ICA (independent component analysis) on the EEG data to obtain the EEG components corresponding to the target EEG data. The specific process of ICA analysis can be referred to formulas (V) to (VIII) below.

[0162] S=(A -1 Formula (5)

[0163] Formula (VI) X = AS

[0164] W = A -1 Formula (VII)

[0165] S = WX Formula (8)

[0166] Where X is the preprocessed target sample EEG data, with the dimension of (number of channels * sampling points); S is the EEG component of the target sample EEG data, with the dimension of (number of leads * sampling points); A is the ICA weight matrix, with the dimension of (number of leads * number of leads); and W is the inverse matrix of matrix A.

[0167] After obtaining the EEG components corresponding to the target sample EEG data, an EEG topography map can be drawn based on the EEG components corresponding to the target sample EEG data. Based on prior knowledge, the electrooculography (EOG) components are selected from the EEG components corresponding to the target sample EEG data, and then an EOG template is constructed based on the selected EOG components.

[0168] Reference Figure 9 As shown, after obtaining the electrooculogram (EOG) template, the target EEG data can be subjected to ICA analysis to obtain the EEG components corresponding to each channel of the target EEG data, and the similarity between the EEG components of each channel and the EOG template can be determined. For example, the correlation coefficient between the EEG components of each channel and the EOG template can be determined as the similarity between the EEG components and the EOG template. The process of determining the similarity coefficient can refer to the following formula (IX).

[0169]

[0170] Where P is used to characterize the correlation coefficient, and Y is used to characterize the EEG components corresponding to the target EEG data.

[0171] After performing EEG artifact removal on the target computer data, channels with similarity greater than the similarity threshold are removed, while channels with similarity less than or equal to the similarity threshold are retained. For the retained channels, the EEG components of the retained channels are restored to EEG data to obtain the artifact-removed target EEG data for the next step of feature extraction. The specific method for restoring EEG data can be referred to in the following formulas (viii) and (ix).

[0172] Based on prior knowledge, the target EEG components for artifact removal can be used as a new component matrix S'. During the EEG data recovery process, components corresponding to the electrooculogram (EOG) components in the ICA weight matrix are simultaneously removed, resulting in a new matrix W'. The artifact-removed target EEG data can then be obtained using the formula X = W'S'.

[0173] Based on the EEG data processing method provided in this disclosure, an EEG template can be constructed using sample EEG data corresponding to multiple sampling objects. Then, based on the EEG template, EEG artifact removal processing can be performed on the target EEG data, which can improve the accuracy of EEG artifact removal and thus improve the prediction accuracy of the target object's emotion category.

[0174] In one embodiment, the current emotional state may further include focus, as referenced Figure 10 and Figure 11As shown, in step 206, determining the current emotional state of the target subject based on the target EEG data may further include:

[0175] Step 1002: Divide the data in each channel of the target EEG data into windows to obtain multiple windows;

[0176] Step 1004: Determine the blink window from multiple windows, wherein the standard deviation of the EEG data within the blink window is greater than the standard deviation threshold;

[0177] Step 1006: Determine the target's level of focus at the current moment based on the total number of blink windows and the total number of channels of the target EEG data.

[0178] In this embodiment of the disclosure, the channels of the target EEG data can be divided into windows according to a preset data volume to obtain multiple windows. For example, taking the aforementioned example, the data dimension of the target EEG data is 3×1250. Assuming the preset data volume is set to 125, for any channel, the EEG data of that channel can be set as 1 window with 125 data points. Each channel is divided into 10 windows, for a total of 3 channels, so the target EEG data is divided into a total of 30 windows.

[0179] A blinking window can be identified from multiple windows. For example, for any given window, the standard deviation of that window can be determined. If the standard deviation is greater than a preset standard deviation threshold (a preset value; this embodiment does not specify a specific value, but the specific value can be set by those skilled in the art according to their needs), the window can be identified as a blinking window.

[0180] The total number of blink windows within the window (i.e., the total number of blinks by the target subject during the sampling time) can be determined. Based on the ratio of the total number of blink windows to the total number of channels in the target EEG data, the target subject's level of focus at the current moment can be determined. When the focus level is a value of 1-100, the ratio of the total number of blink windows to the total number of channels in the target EEG data is negatively correlated with the corresponding focus level value. That is, the larger the ratio, the more frequently the target subject blinks, and thus the lower the target subject's focus level, and the smaller the corresponding focus level value.

[0181] Based on the EEG data processing method provided in this disclosure, the blink count of the target object can be analyzed through the target EEG data of the target object, and then the focus of the target object can be determined based on the blink count, which can improve the accuracy of the focus of the target object.

[0182] In one embodiment, reference Figure 12As shown, in step 1006, determining the target's level of focus at the current moment based on the total number of blink windows and the total number of channels in the target EEG data may include:

[0183] Step 1202: Determine the initial level of focus of the target object at the current moment based on the total number of blink windows and the total number of channels of the target EEG data;

[0184] Step 1204: Smooth the historical and initial focus levels of the target object within the first preset time period to obtain the focus level of the target object at the current moment.

[0185] In this embodiment, the target's current level of focus can be determined as the initial level of focus based on the total number of blink windows and the total number of channels in the target EEG data. After determining the target's initial level of focus, the target's historical focus within a first preset time period (a preset time interval from the current time) can be obtained. The initial level of focus is then smoothed based on the historical focus to obtain the target's current level of focus. For example, the focus within a 5-second period (including historical focus and the initial level of focus at the current moment) can be smoothed to obtain the target's current level of focus.

[0186] After obtaining the target's current level of focus, the display interface can show the target's current level of focus, so that users can know the target's current level of focus in real time, and then evaluate the business based on the target's current level of focus or other evaluation dimensions.

[0187] For example, refer to Figure 5 As shown, the display interface includes a focus display area, which displays the focus of the target object at the current moment. This focus display area can display the numerical value or level of the focus, as well as the change curve of the focus over the monitoring period. In this embodiment, no specific limitation is made on the display method of focus.

[0188] Based on the EEG data processing method provided in this embodiment, the initial focus can be smoothed by the historical focus within a first preset time period, thereby improving the accuracy of the target's focus.

[0189] In one embodiment, the current emotional state also includes an emotional change trajectory, as referenced Figure 13 As shown, in step 206, determining the current emotional state of the target subject based on the target EEG data may further include:

[0190] Step 1302: Based on the target object's current emotion category and the target object's current level of focus, determine the target object's coordinate information in the flow coordinate system. The horizontal axis of the flow coordinate system is used to represent the target object's level of focus, and the vertical axis of the flow coordinate system is used to represent the target object's rate of emotional change.

[0191] Step 1304: In the flow trajectory area of ​​the display interface, move the flow marker according to the coordinate information, and obtain the emotional change trajectory corresponding to the target object according to the movement trajectory of the flow marker.

[0192] The Flow Trajectory area includes multiple display areas, each corresponding to a different emotion category.

[0193] In this embodiment of the disclosure, the display interface includes a flow trajectory area, which is used to display the emotional change trajectory of the target object. The flow trajectory area includes a flow coordinate system, which can be based on the center position of the flow trajectory area as the origin. Its horizontal axis can be used to represent the focus of the target object, and its vertical axis can be used to represent the rate of emotional change of the target object.

[0194] The flow trajectory area can be divided into multiple display areas, each corresponding to a different emotion category. Flow markers (also called floating points) are displayed within the flow trajectory area. The location of these flow markers can characterize the target's emotion category, rate of emotion change, and level of focus. For example, when a flow marker is located in display area 1, the target's emotion category can be determined to be the emotion category corresponding to area 1. The larger the absolute value of the horizontal axis of the flow marker's location, the higher the target's level of focus; the larger the absolute value of the vertical axis, the faster the target's rate of emotion change.

[0195] After obtaining the target's current emotion category and level of focus, their coordinates within the flow coordinate system can be determined. Once these coordinates are established, the flow marker can be moved from its previous position to the new location. Based on the marker's movement during monitoring, the target's emotional trajectory can then be derived.

[0196] Based on the EEG data processing method provided in this disclosure, the target object's coordinate information in the flow coordinate system can be determined by the target object's current emotion category and the target object's current concentration level. Then, based on the coordinate information, the flow marker is moved in the flow trajectory area to obtain the target object's emotion change trajectory. This allows users to know the target object's emotion change status in real time based on the emotion change trajectory, and to evaluate or adjust the business in real time based on the emotion change trajectory.

[0197] In one embodiment, reference Figure 5 As shown, the flow trajectory region is divided into four display areas by the four quadrants of the flow coordinate system. The four display areas include: the first quadrant of the flow coordinate system corresponds to the first emotion and the first display area; the second quadrant of the flow coordinate system corresponds to the second emotion and the second display area; the third quadrant of the flow coordinate system corresponds to the third emotion and the third display area; and the fourth quadrant of the flow coordinate system corresponds to the fourth emotion and the fourth display area.

[0198] In this embodiment of the disclosure, the flow trajectory region can be divided into four display areas using the four quadrants of the flow coordinate system. These four display areas can correspond to four emotion categories. For example: the first quadrant can correspond to the first emotion and the first display area; the first emotion can be relaxation, meaning the emotion category corresponding to the first display area is relaxation. The second quadrant can correspond to the second emotion and the second display area; the second emotion can be excitement, meaning the emotion category corresponding to the second display area is excitement. The third quadrant corresponds to the third emotion and the third display area; the third emotion can be boredom, meaning the emotion category corresponding to the third display area is boredom. The fourth quadrant corresponds to the fourth emotion and the fourth display area; the fourth display area can be anxiety, meaning the emotion category corresponding to the fourth display area is anxiety.

[0199] Based on the EEG data processing method provided in this disclosure, the emotional change trajectory of the target object can be obtained by the movement of the flow marker, so that the user can know the emotional change of the target object in real time based on the emotional change trajectory, and make real-time evaluation or adjustment of the business based on the emotional change trajectory.

[0200] In one embodiment, reference Figure 14 As shown, in step 1302, the coordinate information of the target object in the flow coordinate system is determined based on the target object's current emotion category and the target object's current level of focus. This may include:

[0201] Step 1402: Determine the x-coordinate of the target object in the flow coordinate system based on the target object's current emotion category and the target object's current level of focus.

[0202] Step 1404: Determine the vertical coordinate of the target object in the flow coordinate system based on the difference between the two emotion categories of the target object at adjacent times within the second preset time period.

[0203] In this embodiment of the disclosure, when the target object's current emotion category corresponds to either the emotion category in the first quadrant or the emotion category in the fourth quadrant, the horizontal coordinate of the target object in the flow coordinate system is negatively correlated with the target object's current level of focus. For example, taking a focus level of 1 to 100, the horizontal coordinate X of the target object in the flow coordinate system is -focus level + 100.

[0204] When the target's current emotional category corresponds to either the second or third quadrant, the x-coordinate of the target in the flow coordinate system is positively correlated with the target's current level of focus. For example, using a focus level of 1 to 100, the target's x-coordinate in the flow coordinate system would be X = focus level - 100.

[0205] It can obtain the emotional category of the target object at various moments within a second preset time period before the current moment (a time period with a preset interval from the current moment; the second preset time period can be the same as or different from the first preset time period). For example, it can take the emotional category 5 seconds before the current moment and combine it with the emotional category at the current moment to form an emotional category set.

[0206] For any two emotion categories at any adjacent time within the emotion category set, if the two emotion categories are identical, their ratio can be determined to be 1; otherwise, their ratio can be determined to be 0. Furthermore, by summing the ratios of emotion categories at any two adjacent time points within the emotion category set, and considering the total number of emotion categories within the second preset time period, the rate of change of the target object's emotions can be determined, which is equivalent to determining the target object's ordinate in the flow coordinate system.

[0207] When the target's current emotion category corresponds to either the first or second quadrant, the sum of the ratios of the target's ordinate in the flow coordinate system to the emotion categories at two adjacent moments within the emotion category set is negatively correlated. When the target's current emotion category corresponds to either the third or fourth quadrant, the sum of the ratios of the target's ordinate in the flow coordinate system to the emotion categories at two adjacent moments within the emotion category set is positively correlated.

[0208] For example, when the target object's emotion category corresponds to either the first or second quadrant, the ordinate of the target object in the flow coordinate system is the difference obtained by subtracting the sum of the total number of emotion categories within the second preset time period from the ratio of the emotion categories at any two adjacent moments within the emotion category set. When the target object's emotion category corresponds to either the third or fourth quadrant, the ordinate of the target object in the flow coordinate system is the difference obtained by subtracting the sum of the ratios of the emotion categories at any two adjacent moments within the emotion category set from the total number of emotion categories within the second preset time period.

[0209] Taking a second preset duration of 5 seconds as an example, the target's emotional category at the current moment is represented as M0, and the emotional categories from 1 second to 5 seconds before the current moment are M1, M2, M3, M4, and M5, respectively. We can determine M0 / M1, M1 / M2, M2 / M3, M3 / M4, and M4 / M5 respectively. When M0 corresponds to the emotional category in the first quadrant or the second quadrant, the total number of emotional categories within the second preset time period is 5. Therefore, the ordinate Y of the target in the flow coordinate system is -(M0 / M1 + M1 / M2 + M2 / M3 + M3 / M4 + M4 / M5) + 5. Alternatively, when M0 corresponds to the emotional category in the third quadrant or the fourth quadrant, the ordinate Y of the target in the flow coordinate system is (M0 / M1 + M1 / M2 + M2 / M3 + M3 / M4 + M4 / M5) - 5.

[0210] Reference Figure 15 As shown, after obtaining the target object's current coordinate information (X1, Y1) in the flow coordinate system, the flow marker can be moved from the previous position (X0, Y0) to (X1, Y1). After obtaining the target object's next coordinate information (X2, Y2) in the flow coordinate system, the flow marker can be moved from the position (X1, Y1) to (X2, Y2).

[0211] In one example, after obtaining the coordinate information (X1, Y1) of the target object in the flow coordinate system at the current moment, the coordinate information (X1, Y1) can be further transformed into coordinate information in the flow trajectory area of ​​the display interface. Then, the flow marker is moved in the flow trajectory area according to the coordinate information in the flow trajectory area to obtain the emotional change trajectory corresponding to the target object.

[0212] The absolute value of the x-coordinate of the target object's coordinate information in the flow trajectory region can be the absolute value of the logarithm of X1. Based on the quadrant in which the target object's current emotion category is located, the x-coordinate of the target object's coordinate information in the flow trajectory region can be obtained. For example, when the target object's current emotion category corresponds to the first or fourth quadrant, the x-coordinate of the target object's coordinate information in the flow trajectory region is the absolute value of the logarithm of X1; when the target object's current emotion category corresponds to the second or third quadrant, the x-coordinate of the target object's coordinate information in the flow trajectory region is -(the absolute value of the logarithm of X1).

[0213] The absolute value of the ordinate of the target object's coordinate information in the flow trajectory region can be the absolute value of the logarithm of the Y value, where the Y value is the average of the target object's ordinates in the flow coordinate system over a preset time period. This preset time period can be a pre-defined period, and its specific value is not limited. For example, the preset time period could be within 3 seconds, meaning the ordinates of the target object's ordinates 1 second and 2 seconds prior to the current moment are averaged with the current Y1 to obtain the Y value. Furthermore, based on the quadrant in which the target object's current emotion category falls, the ordinate of the target object's coordinate information in the flow trajectory region can be obtained. For example, when the target object's current emotion category corresponds to the first or second quadrant, the abscissa of the target object's coordinate information in the flow trajectory region is the absolute value of the logarithm of Y; when the target object's current emotion category corresponds to the third or fourth quadrant, the abscissa of the target object's coordinate information in the flow trajectory region is -(the absolute value of the logarithm of Y).

[0214] By moving the flow markers according to the coordinate information in the flow trajectory area, the resulting emotional change trajectory of the target object can be displayed more prominently in the display interface.

[0215] Based on the EEG data processing method provided in this disclosure, the target object's coordinate information in the flow coordinate system can be determined by the target object's current emotion category and the target object's current concentration level. Then, the flow marker is moved according to the coordinate information to obtain the target object's emotional change trajectory. This allows users to know the target object's emotional changes in real time based on the emotional change trajectory, and to evaluate or adjust the business in real time based on the emotional change trajectory.

[0216] In one embodiment, an energy marker is displayed at the location where the center flow marker of the emotion change trajectory has stopped, and the size and / or brightness of the energy marker is negatively correlated with the display duration of the energy marker.

[0217] In this embodiment of the disclosure, after the flow marker moves from the current position to the next position, a corresponding energy marker can be generated and displayed at the current position. The size and / or brightness of the energy marker are negatively correlated with the display duration of the energy marker. That is, as the display duration of the energy marker increases, the size gradually decreases and / or the brightness gradually dims.

[0218] Reference Figure 5 As shown, the energy markers in the emotional change trajectory displayed in the flow trajectory area vary in size and brightness; the longer the display time, the dimmer and smaller the energy markers. Based on this, users can clearly understand the emotional changes of the target object within the monitoring period, enabling them to better conduct business evaluations based on these emotional changes.

[0219] To enable those skilled in the art to better understand the embodiments of this disclosure, the embodiments of this disclosure are described below through specific examples.

[0220] The EEG data processing method provided in this disclosure can be applied to teaching scenarios, see below. Figure 1b As shown in the diagram, in this scenario, students can wear EEG data acquisition devices during the lecture. The instructor can determine the student's current listening status based on the displayed emotional state, which can then reflect the quality of the instructor's teaching. This allows for adaptive adjustments to the teaching content or methods. Alternatively, the instructor can determine the student's level of comprehension of the lecture content based on their current emotional state, enabling targeted and personalized tutoring.

[0221] For example, refer to Figure 5 As shown, the display interface includes a monitoring object display area, which displays a list of monitored students. This list may include students' personal information, monitoring duration, and other information. The display interface also includes start and end controls. The terminal can respond to a trigger operation on the start control to begin determining the current emotional state of the monitored student based on the monitored student's EEG data, or the terminal can respond to a trigger operation on the end control to end the analysis and prediction of the monitored student's current emotional state.

[0222] The instructor can select a target student from the students displayed in the monitoring area. The terminal then responds to this selection by displaying the target student's current emotional state on the interface. This includes the waveform of the target student's EEG data, the current emotion category, the trajectory of emotional changes, the current level of focus, the proportion of each emotion category during the monitoring period, and the EEG components. Based on this displayed emotional state, the instructor can determine the target student's current engagement during the lesson.

[0223] Alternatively, the display interface can show a distribution chart of all monitored students' emotions. This distribution chart can show the distribution of all monitored students' emotions at the same time, or the distribution of all monitored students' emotions during the monitoring period. Based on this distribution chart, authorized teachers can evaluate the quality of the current teaching content in real time.

[0224] For example, if the display interface indicates that students' overall concentration is high and they are generally excited and relaxed, then it can be determined that the teacher's teaching quality is good; or, if the display interface indicates that students' overall concentration is low and they are relatively anxious or bored, then it can be determined that the teacher's teaching quality may need to be improved.

[0225] Alternatively, the EEG data processing method provided in this embodiment can be applied to gaming scenarios. If the display interface indicates that the player's overall focus is high, and they are generally excited and relaxed, it means that the current game difficulty matches the player's skill level, and the player's gaming experience is good. If the display interface indicates that the player's overall focus is low, and they are relatively anxious or bored, it means that the player's gaming experience is average, and the game's difficulty or content may need to be improved.

[0226] Alternatively, the EEG data processing method provided in this embodiment can be applied to the field of engineering safety. It can analyze the working status of relevant operators by displaying the current emotional state of the target object on the display interface, and then provide corresponding feedback to ensure the safety of the operators.

[0227] The above-mentioned fields are merely examples of application fields of the embodiments of this disclosure. In fact, the embodiments of this disclosure can be applied to any field that can evaluate business or operations based on user emotions, and the embodiments of this disclosure do not specifically limit this.

[0228] In one embodiment, as shown in FIG1, an electroencephalogram (EEG) data processing system is provided, the system including an EEG data acquisition device 102, a data processing device 104, and a display device 106; wherein...

[0229] EEG data acquisition 102 is used to acquire EEG data of the target object and send the EEG data of the target object to the data processing device;

[0230] The data processing device 104 is used to preprocess the EEG data of the target object, and after obtaining the target EEG data, determine the current emotional state of the target object based on the target EEG data.

[0231] Display device 106 is used to display the current emotional state of the target object in a display interface in response to a viewing operation on the target object.

[0232] In this embodiment, the data interaction and processing of EEG data between the EEG data acquisition device 102, the data processing device 104, and the display device 106 are similar to the EEG data processing process in the aforementioned embodiments. Therefore, this embodiment will not repeat the details here, but you can refer to the relevant descriptions in the aforementioned embodiments.

[0233] Based on the EEG data processing system provided in this disclosure, the current emotional state of the target object can be obtained in real time through the target object's EEG data and displayed accordingly. This allows users to obtain the target object's emotional state in real time and then evaluate the business in real time based on the target object's emotional state. Since the emotional state can more realistically represent the user's feelings and is not affected by the target object's subjective consciousness, it can not only improve the accuracy of business evaluation but also make the real-time performance of business evaluation better.

[0234] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0235] Based on the same inventive concept, embodiments of the present invention also provide an electroencephalogram (EEG) data processing device for implementing the aforementioned EEG data processing method. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more EEG data processing device embodiments provided below can be found in the limitations of the EEG data processing method described above, and will not be repeated here.

[0236] In one embodiment, such as Figure 16 As shown, an EEG data processing device is provided, comprising: an acquisition module 1602, a preprocessing module 1604, a first determination module 1606, and a display module 1608, wherein:

[0237] Module 1602 is used to acquire the EEG data of the target object;

[0238] Preprocessing module 1604 is used to perform preprocessing operations on the EEG data to obtain target EEG data;

[0239] The first determining module 1606 is used to determine the current emotional state of the target object based on the target EEG data, wherein the current emotional state is state data associated with the emotion of the target object;

[0240] The display module 1608 is used to display the current emotional state of the target object in the display interface.

[0241] The aforementioned EEG data processing device can preprocess the EEG data of a target object after acquisition to obtain target EEG data, and determine the target object's current emotional state based on the target EEG data. The current emotional state is then displayed on a display interface, where it is state data associated with the target object's emotions. Based on the EEG data processing device provided in this embodiment, the device can acquire and display the target object's current emotional state in real time using the target object's EEG data. This allows users to obtain the target object's emotional state in real time and evaluate business operations accordingly. Since emotional states more realistically represent user feelings and are not influenced by the target object's subjective consciousness, this not only improves the accuracy of business evaluation but also enhances its real-time performance.

[0242] In one embodiment, the current emotional state includes at least one of the following: emotion category, emotional change trajectory, focus level, percentage of each emotion category during the monitoring period, and EEG components of the EEG data.

[0243] In one embodiment, the current emotional state includes an emotional category, and the first determining module 1606 is further configured to:

[0244] Feature extraction is performed on the target EEG data to obtain a first EEG feature and a second EEG feature, wherein the first EEG feature is the time domain feature of the target EEG data and the second EEG feature is the frequency domain feature of the target EEG data;

[0245] An emotion prediction model is used to perform emotion prediction processing on the first EEG feature and the second EEG feature to obtain the emotion category of the target object at the current moment.

[0246] In one embodiment, the device further includes:

[0247] The second determining module is used to determine the similarity between the EEG data corresponding to any channel of the target EEG data and the electrooculogram template.

[0248] The artifact removal module is used to perform EEG artifact removal processing on the target EEG data based on the similarity between the EEG data corresponding to each channel and the EEG template, so as to obtain artifact-removed target EEG data.

[0249] In one embodiment, the device further includes:

[0250] The acquisition module is used to acquire multiple sample EEG data, wherein the sample EEG data is acquired when the sampled object blinks;

[0251] The fusion module is used to fuse the multiple sample EEG data to obtain the target sample EEG data;

[0252] The analysis module is used to perform component analysis on the target sample EEG data to obtain the EEG components corresponding to the target sample EEG data;

[0253] An extraction module is used to extract electrooculography (EOG) components from the electroencephalogram (EEG) components to obtain the EOG template.

[0254] In one embodiment, the current emotional state further includes focus, and the first determining module 1606 is further configured to:

[0255] The data in each channel of the target EEG data is divided into multiple windows;

[0256] A blink window is determined from the plurality of windows, wherein the standard deviation of the EEG data within the blink window is greater than a standard deviation threshold;

[0257] The focus level of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

[0258] In one embodiment, the first determining module is further configured to:

[0259] The initial level of focus of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

[0260] The historical focus level and initial focus level of the target object within a first preset time period are smoothed to obtain the focus level of the target object at the current moment.

[0261] In one embodiment, the current emotional state further includes an emotional change trajectory, and the first determining module 1606 is further configured to:

[0262] Based on the target object's current emotion category and the target object's current level of focus, determine the target object's coordinate information in the flow coordinate system. The horizontal axis of the flow coordinate system is used to represent the target object's level of focus, and the vertical axis of the flow coordinate system is used to represent the target object's rate of emotion change.

[0263] In the flow trajectory area of ​​the display interface, the flow marker is moved according to the coordinate information, and the emotional change trajectory corresponding to the target object is obtained according to the movement trajectory of the flow marker.

[0264] The flow trajectory area includes multiple display areas, each corresponding to a different emotion category.

[0265] In one embodiment, the flow trajectory region is divided into four display areas by the four quadrants of the flow coordinate system. The four display areas include: the first quadrant of the flow coordinate system corresponds to the first emotion and the first display area; the second quadrant of the flow coordinate system corresponds to the second emotion and the second display area; the third quadrant of the flow coordinate system corresponds to the third emotion and the third display area; and the fourth quadrant of the flow coordinate system corresponds to the fourth emotion and the fourth display area.

[0266] In one embodiment, the first determining module 1606 is further configured to:

[0267] Based on the target object's current emotion category and the target object's current level of focus, determine the target object's horizontal coordinate in the flow coordinate system;

[0268] Based on the difference between two emotion categories of the target object at adjacent times within a second preset time period, the vertical coordinate of the target object in the flow coordinate system is determined.

[0269] In one embodiment, an energy marker is displayed at the location where the flow marker in the emotion change trajectory has stayed, and the size and / or brightness of the energy marker is negatively correlated with the display duration of the energy marker.

[0270] Each module in the aforementioned EEG data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0271] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 17 As shown, the computer device includes a processor, memory, communication interface, display screen, and input device connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a brainwave data processing method. The display screen can be an LCD screen or an e-ink screen. The input device can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0272] Those skilled in the art will understand that Figure 17 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device to which the present invention is applied. A specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0273] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0274] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0275] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0276] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this invention are all information and data authorized by the user or fully authorized by all parties.

[0277] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided by this invention can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided by this invention may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided by this invention may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0278] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For system embodiments, since they largely correspond to method embodiments, the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0279] The methods and systems of the present invention may be implemented in many ways. For example, they may be implemented by software, hardware, firmware, or any combination of software, hardware, and firmware. The above-described order of steps for the methods is for illustrative purposes only, and the steps of the methods of the present invention are not limited to the order specifically described above unless otherwise specifically stated. Furthermore, in some embodiments, the present invention may also be implemented as a program recorded on a recording medium, the program comprising machine-readable instructions for implementing the methods according to the present invention. Thus, the present invention also covers recording media storing programs for performing the methods according to the present invention.

[0280] The description of this invention is given for illustrative and descriptive purposes only and is not intended to be exhaustive or to limit the invention to the forms disclosed. Many modifications and variations will be apparent to those skilled in the art. The embodiments were chosen and described in order to better illustrate the principles and practical application of the invention and to enable those skilled in the art to understand the invention and to design various embodiments with various modifications suitable for a particular purpose.

Claims

1. A method for processing electroencephalogram (EEG) data, characterized in that, The method includes: Acquire the electroencephalogram (EEG) data of the target subject; The EEG data is preprocessed to obtain the target EEG data; Based on the target EEG data, the current emotional state of the target object is determined, and the current emotional state is state data associated with the target object's emotions; The current emotional state of the target object is displayed in the display interface. The current emotional state includes emotion category, emotional change trajectory, and focus level. Determining the current emotional state of the target object based on the target EEG data includes: Based on the target object's current emotion category and the target object's current level of focus, determine the target object's horizontal coordinate in the flow coordinate system, where the horizontal coordinate of the flow coordinate system is used to characterize the target object's level of focus; An emotion category set is constructed based on the emotion category at the current moment and the emotion categories at each moment within the second preset time period before the current moment. For any two emotion categories at any adjacent moment in the emotion category set, if the two emotion categories are the same, the ratio between them is determined to be 1; otherwise, the ratio between them is determined to be 0. The sum of the ratios of the emotion categories at two adjacent moments in the set of emotion categories is determined. The difference between the total number of emotion categories in the second preset time period and the sum, or the opposite of the difference, is used as the ordinate of the target object in the flow coordinate system. The ordinate of the flow coordinate system is used to characterize the rate of change of the target object's emotions. In the flow trajectory area of ​​the display interface, the flow marker is moved according to the coordinate information. Based on the movement trajectory of the flow marker, the emotional change trajectory corresponding to the target object is obtained. The flow trajectory area is divided into four display areas through the four quadrants of the flow coordinate system, and each display area corresponds to a different emotional category.

2. The method according to claim 1, characterized in that, The current emotional state includes at least one of the following: emotion category, emotional change trajectory, focus level, percentage of each emotion category during the monitoring period, and EEG components of the EEG data.

3. The method according to claim 1 or 2, characterized in that, The current emotional state includes emotion categories. Determining the current emotional state of the target object based on the target EEG data includes: Feature extraction is performed on the target EEG data to obtain a first EEG feature and a second EEG feature, wherein the first EEG feature is the time domain feature of the target EEG data and the second EEG feature is the frequency domain feature of the target EEG data; An emotion prediction model is used to perform emotion prediction processing on the first EEG feature and the second EEG feature to obtain the emotion category of the target object at the current moment.

4. The method according to claim 3, characterized in that, Before performing feature extraction on the target EEG data, the method includes: For any channel of the target EEG data, determine the similarity between the EEG data corresponding to that channel and the electrooculogram template; Based on the similarity between the EEG data corresponding to each channel and the EEG template, the target EEG data is subjected to EEG artifact removal processing to obtain artifact-removed target EEG data.

5. The method according to claim 4, characterized in that, The method further includes: Multiple sample EEG data were collected, wherein the sample EEG data were collected when the sampled subject blinked; The EEG data of the multiple samples are fused to obtain the EEG data of the target sample. Component analysis is performed on the EEG data of the target sample to obtain the EEG components corresponding to the EEG data of the target sample; The electrooculogram (EEG) components were extracted to obtain the electrooculogram template.

6. The method according to claim 3, characterized in that, The current emotional state also includes focus level. Determining the current emotional state of the target object based on the target EEG data further includes: The data in each channel of the target EEG data is divided into multiple windows; A blink window is determined from the plurality of windows, wherein the standard deviation of the EEG data within the blink window is greater than a standard deviation threshold; The focus level of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

7. The method according to claim 6, characterized in that, Determining the target object's level of focus at the current moment based on the total number of blink windows and the total number of channels in the target EEG data includes: The initial level of focus of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data. The historical focus level and initial focus level of the target object within a first preset time period are smoothed to obtain the focus level of the target object at the current moment.

8. The method according to claim 1, characterized in that, The four display areas include: the first quadrant of the flow coordinate system corresponds to the first emotion and the first display area; the second quadrant of the flow coordinate system corresponds to the second emotion and the second display area; the third quadrant of the flow coordinate system corresponds to the third emotion and the third display area; and the fourth quadrant of the flow coordinate system corresponds to the fourth emotion and the fourth display area.

9. The method according to claim 1, characterized in that, Energy markers are displayed at the locations where the flow markers have lingered in the emotional change trajectory, and the size and / or brightness of the energy markers are negatively correlated with the display duration of the energy markers.

10. A brainwave data processing system, characterized in that, The system includes an electroencephalogram (EEG) data acquisition device, a data processing device, and a display device; wherein... The EEG data acquisition is used to collect the EEG data of the target object and send the EEG data of the target object to the data processing device; The data processing device is used to preprocess the EEG data of the target object, and after obtaining the target EEG data, determine the current emotional state of the target object based on the target EEG data. The display device is configured to, in response to a viewing operation on the target object, display the current emotional state of the target object in a display interface. The current emotional state includes emotion category, emotional change trajectory, and focus level. Determining the current emotional state of the target object based on the target EEG data includes: Based on the target object's current emotion category and the target object's current level of focus, determine the target object's horizontal coordinate in the flow coordinate system, where the horizontal coordinate of the flow coordinate system is used to characterize the target object's level of focus; An emotion category set is constructed based on the emotion category at the current moment and the emotion categories at each moment within the second preset time period before the current moment. For any two emotion categories at any adjacent moment in the emotion category set, if the two emotion categories are the same, the ratio between them is determined to be 1; otherwise, the ratio between them is determined to be 0. The sum of the ratios of the emotion categories at two adjacent moments in the set of emotion categories is determined. The difference between the total number of emotion categories in the second preset time period and the sum, or the opposite of the difference, is used as the ordinate of the target object in the flow coordinate system. The ordinate of the flow coordinate system is used to characterize the rate of change of the target object's emotions. In the flow trajectory area of ​​the display interface, the flow marker is moved according to the coordinate information. Based on the movement trajectory of the flow marker, the emotional change trajectory corresponding to the target object is obtained. The flow trajectory area is divided into four display areas through the four quadrants of the flow coordinate system, and each display area corresponds to a different emotional category.

11. A brainwave data processing device, characterized in that, The device includes: The acquisition module is used to acquire the EEG data of the target object; The preprocessing module is used to perform preprocessing operations on the EEG data to obtain the target EEG data; The first determining module is used to determine the current emotional state of the target object based on the target EEG data, wherein the current emotional state is state data associated with the emotion of the target object; The display module is used to display the current emotional state of the target object in the display interface. The current emotional state includes emotion category, emotional change trajectory, and focus level. Determining the current emotional state of the target object based on the target EEG data includes: Based on the target object's current emotion category and the target object's current level of focus, determine the target object's horizontal coordinate in the flow coordinate system, where the horizontal coordinate of the flow coordinate system is used to characterize the target object's level of focus; An emotion category set is constructed based on the emotion category at the current moment and the emotion categories at each moment within the second preset time period before the current moment. For any two emotion categories at any adjacent moment in the emotion category set, if the two emotion categories are the same, the ratio between them is determined to be 1; otherwise, the ratio between them is determined to be 0. The sum of the ratios of the emotion categories at two adjacent moments in the set of emotion categories is determined. The difference between the total number of emotion categories in the second preset time period and the sum, or the opposite of the difference, is used as the ordinate of the target object in the flow coordinate system. The ordinate of the flow coordinate system is used to characterize the rate of change of the target object's emotions. In the flow trajectory area of ​​the display interface, the flow marker is moved according to the coordinate information. Based on the movement trajectory of the flow marker, the emotional change trajectory corresponding to the target object is obtained. The flow trajectory area is divided into four display areas through the four quadrants of the flow coordinate system, and each display area corresponds to a different emotional category.

12. The apparatus according to claim 11, characterized in that, The current emotional state includes at least one of the following: emotion category, emotional change trajectory, focus level, percentage of each emotion category during the monitoring period, and EEG components of the EEG data.

13. The apparatus according to claim 11 or 12, characterized in that, The current emotional state includes emotional categories, and the first determining module is further configured to: Feature extraction is performed on the target EEG data to obtain a first EEG feature and a second EEG feature, wherein the first EEG feature is the time domain feature of the target EEG data and the second EEG feature is the frequency domain feature of the target EEG data; An emotion prediction model is used to perform emotion prediction processing on the first EEG feature and the second EEG feature to obtain the emotion category of the target object at the current moment.

14. The apparatus according to claim 13, characterized in that, The device further includes: The second determining module is used to determine the similarity between the EEG data corresponding to any channel of the target EEG data and the electrooculogram template. The artifact removal module is used to perform EEG artifact removal processing on the target EEG data based on the similarity between the EEG data corresponding to each channel and the EEG template, so as to obtain artifact-removed target EEG data.

15. The apparatus according to claim 14, characterized in that, The device further includes: The acquisition module is used to acquire multiple sample EEG data, wherein the sample EEG data is acquired when the sampled object blinks; The fusion module is used to fuse the multiple sample EEG data to obtain the target sample EEG data; The analysis module is used to perform component analysis on the target sample EEG data to obtain the EEG components corresponding to the target sample EEG data; An extraction module is used to extract electrooculography (EOG) components from the electroencephalogram (EEG) components to obtain the EOG template.

16. The apparatus according to claim 13, characterized in that, The current emotional state also includes focus level, and the first determining module is further used for: The data in each channel of the target EEG data is divided into multiple windows; A blink window is determined from the plurality of windows, wherein the standard deviation of the EEG data within the blink window is greater than a standard deviation threshold; The focus level of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data.

17. The apparatus according to claim 16, characterized in that, The first determining module is further configured to: The initial level of focus of the target object at the current moment is determined based on the total number of blink windows and the total number of channels of the target EEG data. The historical focus level and initial focus level of the target object within a first preset time period are smoothed to obtain the focus level of the target object at the current moment.

18. The apparatus according to claim 11, characterized in that, The four display areas include: the first quadrant of the flow coordinate system corresponds to the first emotion and the first display area; the second quadrant of the flow coordinate system corresponds to the second emotion and the second display area; the third quadrant of the flow coordinate system corresponds to the third emotion and the third display area; and the fourth quadrant of the flow coordinate system corresponds to the fourth emotion and the fourth display area.

19. The apparatus according to claim 11, characterized in that, Energy markers are displayed at the locations where the flow markers have lingered in the emotional change trajectory, and the size and / or brightness of the energy markers are negatively correlated with the display duration of the energy markers.

20. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 9.

21. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

22. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 9.

Images