Intelligent emotion recognition and pacifying system based on brain-computer interface and use method thereof

Abstract

The invention discloses an intelligent emotion recognition and pacifying system based on a brain-computer interface, and aims to solve the emotion problem caused by pressure of people in the modern society. According to the system, by integrating a brain-computer interface technology, a signal processing algorithm, an artificial intelligence technology and a personalized service strategy, accurate recognition and effective pacification of user emotion are realized. The brain-computer interface device collects electroencephalogram signals, the electroencephalogram signals are processed, an SVM support vector machine algorithm is used for analysis, and the negative emotion state is accurately recognized. When negative emotions are detected, the system provides AI pet interactive pacifying and pushes personalized pacifying content, such as music, movie recommendation and emotional diary records, and the system visually analyzes the emotions of the user per week. The method has a wide application prospect in the fields of learning, working places, personal life and the like, and can help the user to better understand and manage own emotions, thereby improving life quality and working efficiency.

Description

Technical Field

[0001] The present invention relates to the fields of human-computer interaction, psychology, computer science and data processing, and specifically refers to an intelligent emotion recognition and soothing system based on a brain-computer interface and a method of using the system. Background Art

[0002] The rapid development of modern society and the accelerated pace of life have gradually increased the pressure people face. Studies have shown that work burnout, academic pressure and life events may cause emotional problems such as anxiety and depression, which not only affect the individual's mental health, but may also have a negative impact on their social functions.

[0003] Traditional methods of emotion management include self-regulation, psychological counseling, and social support, but they have certain limitations. Self-regulation requires individuals to have high psychological literacy and self-control, and the effect varies from person to person; psychological counseling requires appointments with professionals for interviews, which is expensive and inconvenient to solve emotional problems in real time. Social support, such as talking to others and engaging in recreational activities, is limited by time, occasion, and object.

[0004] In recent years, the development of brain-computer interface technology (BCI) has provided a new path for real-time recognition and management of emotions. By collecting bioelectric signals from the brain (such as EEG), BCI technology can directly obtain neural activity data related to emotions, and analyze these data in combination with artificial intelligence algorithms to achieve automatic recognition of emotional states, which opens up new possibilities for precise emotion management. However, the current BCI-based emotion management applications on the market are still in their infancy, and most products are limited to emotion recognition and lack personalized emotion soothing solutions. To this end, it is particularly necessary to develop a system that integrates brain-computer interface, emotion recognition and personalized soothing. Summary of the invention

[0005] The technical problem to be solved by the present invention is to overcome the above technical defects and provide an intelligent emotion recognition and soothing system based on brain-computer interface, break through the limitations of traditional emotion management methods, achieve seamless connection from emotion recognition to soothing, provide users with personalized soothing, and achieve the purpose of soothing the soul.

[0006] In order to solve the above technical problems, the technical solution provided by the present invention is: an intelligent emotion recognition and soothing system based on brain-computer interface, including an EEG signal acquisition module, an EEG signal processing module, an emotion recognition and analysis module, a soothing module, and a data storage and analysis module;

[0007] The EEG signal acquisition module collects emotional EEG signals from the user's brain, the EEG signal processing module analyzes the emotional EEG signals, and the emotion recognition and analysis module classifies the input emotional EEG signals and identifies the types of emotions;

[0008] The soothing module obtains emotion type information for soothing, and the data storage and analysis module records the user's emotional state and usage frequency.

[0009] Preferably, the EEG signal acquisition module includes an EEG electrode cap unit.

[0010] Preferably, the EEG signal processing module comprises the following steps:

[0011] Step 1: Preprocess the raw data;

[0012] Step 2: Use the Fast ICA algorithm to separate the EEG signal and remove artifact components based on sample entropy;

[0013] Step 3: Reconstruct the signal of the independent components after artifact removal to obtain the pure EEG data after artifact removal.

[0014] Preferably, the emotion recognition and analysis module recognizes emotional EEG signals based on a support vector machine and classifies emotions based on a two-dimensional emotion model.

[0015] Preferably, the soothing module simulates pet movement based on Unity and llm applications.

[0016] Preferably, the data storage and analysis module uses statistical graphics to display data information, and the statistical graphics include bar graphs and line graphs.

[0017] Another aspect of the present invention discloses a method for using the system, comprising the following steps:

[0018] Step 1: Wear the brain-computer interface helmet correctly on your head, adjust the headband to a comfortable and fitting position, and ensure that the EEG electrode cap is in full contact with the scalp;

[0019] Step 2: Turn on the power of the device that comes with the helmet, and use the RFDuino radio module and OpenBCI USB Bluetooth communication module to achieve wireless connection between the helmet and the computer and mobile phone APP;

[0020] Step 3: Collect the user's brain wave data and use the SVM support vector machine algorithm to quickly process and analyze the data to identify the user's emotional state, including anxiety, impatience, frustration, and sadness, and display the emotional information on the mobile phone APP;

[0021] Step 4: When negative emotions are detected, the mobile app pops up a cartoon pet image to guide users to choose a soothing method

[0022] Preferably, soothing methods include chatting with a desk pet AI, music, book or movie recommendations, and writing a mood diary.

[0023] The advantages of the present invention compared with the prior art are:

[0024] In human-computer interaction, brain-computer interface communication links and optimized interfaces are constructed. With the help of psychological principles, soothing functional services are designed. Computer science drives software and algorithm development and cloud computing applications. Data processing ensures data collection, analysis, security and privacy protection. In the form of desk pets, users are provided with chat companionship and interaction based on the purpose of emotional soothing, so as to achieve accurate emotion recognition and personalized soothing.

[0025] It has broad application prospects in the fields of learning, workplace and personal life, and can help users better understand and manage their emotions, thereby improving the quality of life and work efficiency. At the same time, SmartSensor focuses on user experience, ensuring that users can easily use and benefit from the system through measures such as comfortable wearing design, convenient operation interface and data security protection. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is the system module diagram

[0027] Figure 2 Is the system technology roadmap

[0028] Figure 3 It is an EEG electrode cap

[0029] Figure 4 It is Smart Companion——Main interface

[0030] Figure 5 EEG signal preprocessing steps

[0031] Figure 6 Schematic diagram of the VA emotion model. DETAILED DESCRIPTION

[0032] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0033] An intelligent emotion recognition and soothing system based on brain-computer interface, the system consists of two parts: hardware and software. The hardware is an EEG electrode cap 1 equipped with a high-precision EEG signal acquisition module. The software mainly includes four modules: EEG signal processing 2, emotion recognition analysis 3, soothing 4, and data storage and analysis 5. The EEG electrode cap 1 collects emotional EEG EEG signals from the user's brain, the EEG signal processing module 2 analyzes the emotional EEG EEG signals, the emotion recognition analysis module 3 classifies the input emotional EEG signals by establishing a relationship model between emotional EEG signals and emotions, and identifies the types of emotions. The soothing module performs personalized soothing according to the user's identified emotional types, and the data storage and analysis module records the user's emotional state and usage frequency, etc.; wherein:

[0034] The operation of the emotion EEG processing module 2 specifically includes the following steps:

[0035] 1) During implementation, there are usually irrelevant signals such as electrooculography, electromyography and some power frequency interference that cause significant interference to weak EEG signals. In order to obtain purer EEG signals, the original data needs to be preprocessed.

[0036] (1.1) The process of collecting user's EEG signals will be more or less interfered by 50HZ AC, zero drift of equipment components, myoelectricity and eye movement signals. Therefore, it is necessary to filter and reduce noise of the collected raw data. First, the Butterworth filter is used for filtering. The frequency response curve of the Butterworth filter is flat and has good amplitude-frequency characteristics. The expression is as follows:

[0037]

[0038] In addition, in order to minimize the power frequency interference of the 50Hz mains, a notch filter is used for filtering.

[0039] (1.2) This product uses the Fast ICA algorithm to separate EEG signals and remove artifact components based on sample entropy.

[0040] Assuming that the source signal sequence of N sampling points is {s(i)}=s(1),s(2),...,s(N), the calculation steps are as follows:

[0041] (1.2.1) Use the zero mean method to process the original EEG data x(t) and obtain X center (t)

[0042] (1.2.2) In order to make X center The covariance matrix of (t) is the identity matrix. center (t) is whitened to obtain z(t);

[0043] (1.2.3) Assign an initial value to the separation matrix and update the separation matrix W by maximizing the negative entropy j;

[0044] J=(E{G(W T z)}-E{G(v)}) 2

[0045]

[0046] in is the cumulative distribution function, and its probability distribution function is: G'(s) = tanh(a 1 s), take a 1is 1; (1.2.4) The sample entropy of each source signal is calculated by the sample entropy calculation method. The calculation steps are as follows:

[0047] A. Convert the original sequence into an m-dimensional vector S(i), expressed as follows:

[0048] S m (i)={s(i),s(i+1),…,s(i+m-1)},1≤i≤N-m+1

[0049] S m (i) represents a vector consisting of consecutive points from i to i+m-1

[0050] B. Define any two vectors S m (i) and S m (j) The distance d between the two vectors is used to represent the maximum difference (absolute value) between the corresponding points of the two vectors.

[0051]

[0052] C. For each vector S m (i) Calculate all other vectors S whose distance is greater than r m (j) Number B i . Then calculate B i The proportion of total points The expression is as follows:

[0053]

[0054] In this paper, r is set to 0.2*SD(s), where SD(s) is the standard deviation of the source signal.

[0055] D. Under the premise of distance threshold r, calculate B of the entire sequence m (r) value, that is, the value of each vector in the sequence based on (3) Find the average.

[0056]

[0057] E. Change the dimension of the vector to m+1 and calculate B according to steps (2), (3) and (4). (m+1) (r), and finally the sample entropy is calculated as:

[0058]

[0059] (1.2.5) Remove the illegal components in the EEG signal based on the threshold value;

[0060] (1.2.6) Finally, the independent components after artifact removal are reconstructed, X(n) = AS(n), and the pure EEG data after artifact removal is obtained.

[0061] The EEG electrode cap (1) can accurately collect the user's brain wave signals, realize real-time monitoring of brain wave changes, and ensure the timeliness of emotion analysis. The EEG electrode cap adopts advanced encryption algorithms to ensure that the user's brain wave data will not be leaked or abused during transmission, so that the user's privacy is fully protected. The optimized Bluetooth transmission protocol realizes low-latency real-time transmission, so that emotion analysis can be performed instantly. It also supports a variety of operating systems and devices and has strong compatibility. It also has signal filtering and noise reduction functions to eliminate external interference and improve the accuracy of emotion recognition.

[0062] The emotion recognition analysis module (3) recognizes emotional EEG signals based on a support vector machine (SVM) and classifies emotions based on a two-dimensional emotion model, wherein:

[0063] (3.1) For the recognition of emotional EEG signals by support vector machines, support vector machines are mainly used to classify and regress input data or feature vectors. It can be used in any dimension of space, and from different dimensions, SVM can always find something similar to a two-dimensional straight line or a hyperplane to divide the space, so that the classification effect can be achieved. The advantage of SVM is that it has many kernel functions, and can use these kernel functions to deal with high-dimensional problems. At the same time, these kernel functions are also the basis for solving nonlinear problems, and its classification idea is extremely simple, that is, maximizing the distance between the decision surface and the sample. The working principle of the support vector machine is to use the kernel function to project different data into a space and find a hyperplane so that the closest point of two different types of data to the hyperplane is the farthest from the plane.

[0064] For a two-dimensional plane, where w is the weight vector, x is represented as the feature vector, and b is the deviation, SVM finds a hyperplane to achieve optimal classification for the two-dimensional linearly separable data line. The hyperplane is defined as:

[0065] ω T +b=0

[0066] X represents the sample data (x = [x 1 ,x 2 ,...,x n ]), then the distance from any sample point to the hyperplane is:

[0067]

[0068] Each type of data has a support plane, so the distance between two parallel support planes is:

[0069]

[0070] In order to find the optimal decision boundary, we maximize this interval by changing the weights and biases. We generally use the minimum value to represent it. Therefore, the optimization objective function of the support vector machine is:

[0071]

[0072] st y i (ω T x i +b)≥1,i=1,2,...,m

[0073] (3.2) Classify emotions based on the two-dimensional emotion model. Various emotions can be mapped one by one on this model. For example, the emotion of happiness in the discrete model is mapped to a high-valence and high-arousal emotion on the VA emotion model.

[0074] The soothing module (4) uses Unity and LLM technology. Unity's graphics rendering engine can achieve realistic virtual pet images and rich animation effects, allowing users to interact with pets in a virtual environment. And through Unity's built-in physics engine, the pet's movement, reaction and interaction in the environment can be simulated to further enhance the user experience. In addition, the animation system provided by Unity allows developers to create delicate and realistic pet movements, from walking and jumping to actions that interact with users (such as soothing, patting the head, etc.), which can bring real experience to users; LLM technology can fully understand the user's current state and provide users with appropriate chat tone and interaction method decisions.

[0075] AI pets also have a training system that increases user stickiness through pet upgrades, illustrated collections, etc., bringing more fun and challenges to users, and enhancing user participation and sense of accomplishment. The personalized soothing content recommendation system uses advanced knowledge graph technology to deeply mine user data and achieve accurate personalized information recommendations. The data storage and analysis module uses cloud computing or user device processing and analysis, and then presents it to users through network information technology using statistical graphics such as bar charts and line charts. The emotional stabilization and soothing function is designed based on psychological principles, and can provide users with chat companionship and interactive functions based on the purpose of emotional soothing in the form of desk pets based on the user's psychological and emotional results obtained through application statistics.

[0076] When applying:

[0077] Wearing the device: The user first wears the lightweight and breathable brain-computer interface helmet correctly on the head, adjusts the headband to a comfortable and fitting position, and ensures that the EEG electrode cap is in full contact with the scalp to accurately collect brain wave signals.

[0078] System startup and connection: Turn on the power of the device that comes with the helmet, and use the RFDuino radio module and OpenBCI USB Bluetooth communication module to achieve wireless connection between the helmet and the computer and mobile phone APP to ensure smooth data transmission channels.

[0079] Emotion monitoring and analysis: In daily activities, the system will collect the user's brain wave data in real time, and use the SVM support vector machine algorithm to quickly process and analyze the data, accurately identify the user's emotional state, including anxiety, impatience, frustration, sadness and other negative emotions, and display the emotional level and detailed analysis results on the mobile phone APP.

[0080] Soothing function activated: When negative emotions are detected, the mobile APP automatically pops up a cute cartoon pet image to guide users to choose a personalized soothing method. Users can click to chat with the Desk Pet AI, enter their feelings or thoughts, and the Desk Pet will give a warm and empathetic response based on natural language processing technology; they can also browse the music, book or movie recommendation list pushed by the system based on user preferences and emotional state, click to play music or view related books and movie content; they can also write a real-time mood diary to record their inner feelings and help sort out their emotions.

[0081] The intelligent emotion recognition based on brain-computer interface of the present invention can realize the acquisition of EEG signals, accurate recognition and classification of emotions, and soothe the user's emotions and a series of derivative functions. In this regard, the characteristics of the present invention are as follows:

[0082] High-precision emotion recognition: Using advanced EEG electrode cap and sensor technology, combined with SVM support vector machine algorithm, it can accurately capture subtle changes in brain wave signals and accurately identify a variety of negative emotions and their intensity. Compared with traditional emotion recognition methods based on physiological indicators or subjective evaluation, the accuracy is greatly improved.

[0083] Personalized soothing strategies: Based on the user's emotional state and personal preference data, the mobile APP provides a variety of personalized soothing methods. AI table pet chat companionship provides emotional resonance, personalized music, books, and movie recommendations meet the interests of different users, and the emotional diary function assists users in self-reflection and emotional sorting, effectively improving the emotional soothing effect.

[0084] Efficient data transmission and security: The wireless communication link constructed by the RFDuino radio module and the OpenBCI USB Bluetooth communication module ensures that the brain wave data is quickly and stably transmitted to the OpenBCI gui software and mobile phone APP. At the same time, strict data encryption and security protection mechanisms are adopted to effectively prevent data loss or tampering and protect user privacy.

[0085] Comfortable and convenient user experience: The helmet is made of lightweight, breathable materials and is designed with an adjustable headband to ensure the user's comfort for long-term wearing. The mobile APP operation is simple and intuitive, with cartoon image guidance and diversified functional module design, which brings users a convenient and interesting user experience and improves user stickiness.

[0086] To achieve the above object, the technical solution adopted by the present invention is:

[0087] The intelligent emotion recognition and soothing system based on brain-computer interface includes four modules: EEG signal processing, emotion recognition analysis, soothing, and data storage and analysis. The EEG signal processing and emotion recognition analysis module is divided into two parts in terms of hardware and software. The hardware part is worn on the user's head. Its shape and features are mainly designed to collect the user's brain electrical signal waves, highlight vitality and conform to the aesthetics of the times. The software part is allowed to exist in the user's device or remote server, responsible for the analysis and processing of the EEG signals collected by the hardware; the soothing and data storage and analysis module includes the user device end, the data transfer end and the server processing end, providing users with a variety of functional services such as emotion reports and AI pet soothing.

[0088] The EEG signal processing module uses an EEG electrode device equipped with a high-precision EEG signal acquisition module and adopts advanced dry electrode technology to collect the user's brain wave signals in real time, ensuring the stability of signal acquisition while ensuring comfort. In addition, the EEG signal processing module is responsible for preliminary signal preprocessing, preprocessing the collected EEG signals, including filtering noise reduction, artifact removal and segmentation processing, and adopts a combination of Butterworth filter and notch filter to effectively reduce noise interference; the signal preprocessing process of the EEG signal processing module uses Fast ICA algorithm combined with sample entropy calculation to accurately remove artifact components; by reasonably intercepting and dividing signal segments, the data processing process is optimized. The EEG signal processing module extracts features from the preprocessed EEG signals, uses differential entropy technology to divide the EEG signals into different frequency bands, and calculates the differential entropy features of each frequency band to extract key information that effectively represents the emotional state.

[0089] The emotion recognition and analysis module is the core part of the system, responsible for real-time analysis and output of the user's emotional state. The emotion recognition module combines the support vector machine (SVM) algorithm to analyze the extracted features and uses a two-dimensional emotion model to classify emotions, and recognizes and outputs the user's emotional state and its rating indicators in real time, such as anxiety level, happiness level, etc.

[0090] The soothing module can provide real-time feedback to the user on the emotional state identified by the emotion recognition module, and build a statistical chart display function of emotional changes through visual interface, sound prompts, etc. with the help of the user's display device, while providing a variety of emotional stabilization and soothing functions. It also has chat companionship and interactive functions, which are based on LLM technology and can fully understand the user's current state and provide the user with appropriate chat tone and interactive mode decisions.

[0091] The data required by the data storage and analysis module are all based on the user's emotional change data and the user's application usage habits. After being processed and analyzed by cloud computing or user equipment, they are presented to users through network information technology using statistical graphics such as bar charts and line charts. The emotional stabilization and soothing function is designed based on psychological principles, and can provide users with chat companionship and interactive functions based on the purpose of emotional soothing in the form of desk pets based on the user's psychological emotional results statistically obtained by the application.

[0092] The present invention and its embodiments are described above, and such description is not restrictive. The drawings show only one embodiment of the present invention, and the actual structure is not limited thereto. In short, if ordinary technicians in the field are inspired by it, without departing from the purpose of the invention, they can design a structure and embodiment similar to the technical solution without creativity, which should belong to the protection scope of the present invention.

Claims

1. An intelligent emotion recognition and soothing system based on brain-computer interface, characterized by: It includes EEG signal acquisition module, EEG signal processing module, emotion recognition and analysis module, soothing module, data storage and analysis module; The EEG signal acquisition module collects emotional EEG signals from the user's brain, the EEG signal processing module analyzes the emotional EEG signals, and the emotion recognition and analysis module classifies the input emotional EEG signals and identifies the types of emotions; The soothing module obtains emotion type information for soothing, and the data storage and analysis module records the user's emotional state and usage frequency.

2. The intelligent emotion recognition and soothing system based on brain-computer interface according to claim 1 is characterized in that: The electroencephalogram signal acquisition module includes an EEG electrode cap unit.

3. The intelligent emotion recognition and soothing system based on brain-computer interface according to claim 1 or 2, characterized in that: The EEG signal processing module comprises the following steps: Step 1: Preprocess the raw data; Step 2: Use the Fast ICA algorithm to separate the EEG signal and remove artifact components based on sample entropy; Step 3: Reconstruct the signal of the independent components after artifact removal to obtain the pure EEG data after artifact removal.

4. The intelligent emotion recognition and soothing system based on brain-computer interface according to claim 3 is characterized in that: The emotion recognition and analysis module recognizes emotional EEG signals based on a support vector machine and classifies emotions based on a two-dimensional emotion model.

5. The intelligent emotion recognition and soothing system based on brain-computer interface according to claim 1 is characterized in that: The soothing module simulates pet movement based on Unity and llm application.

6. The intelligent emotion recognition and soothing system based on brain-computer interface according to claim 1 is characterized in that: The data storage and analysis module uses statistical graphics to display data information, and the statistical graphics include bar graphs and line graphs.

7. A method for using the system, applied to the intelligent emotion recognition and soothing system based on brain-computer interface according to any one of claims 1 to 6, characterized in that: The steps include: Step 1: Wear the brain-computer interface helmet correctly on your head, adjust the headband to a comfortable and fitting position, and ensure that the EEG electrode cap is in full contact with the scalp; Step 2: Turn on the power of the device that comes with the helmet, and use the RFDuino radio module and OpenBCI USB Bluetooth communication module to achieve wireless connection between the helmet and the computer and mobile phone APP; Step 3: Collect the user's brain wave data and use the SVM support vector machine algorithm to quickly process and analyze the data to identify the user's emotional state, including anxiety, impatience, frustration, and sadness, and display the emotional information on the mobile phone APP; Step 4: When negative emotions are detected, the mobile app pops up a cartoon pet image to guide users to choose a soothing method.

8. A method for using the system according to claim 7, characterized in that: The soothing methods include chat companionship with desk pet AI, music, book or movie recommendations, and writing a mood diary.

Images