Emotion recognition and neuromodulation system and method based on bilateral amygdala phase difference

Abstract

This disclosure relates to an emotion recognition and neural modulation system and method based on bilateral amygdala phase difference, comprising: a physiological signal acquisition device for acquiring a user's physiological signals; a facial expression acquisition device for acquiring the user's facial expressions and eye movement information; an electroencephalogram (EEG) device for acquiring the user's electroencephalogram (EEG) signals; a magnetic resonance imaging (MRI) device for acquiring the user's brain activity map; and a data processing module for preprocessing the physiological signals, facial expressions, and eye movement information before inputting them into an emotion recognition model to obtain an emotion category, determining neural activity patterns based on the EEG signals and brain activity map using a neural network model, and adjusting the operating parameters of the emotion recognition model, the EEG device, and the MRI device based on the comparison results of the neural activity patterns and the emotion category. Using the above technical solution, high-precision monitoring and regulation of emotional states and neural activity are achieved.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, and in particular to an emotion recognition and neural modulation system and method based on bilateral amygdala phase difference. Background Technology

[0002] Current emotion recognition technologies and neural modulation systems face challenges. In emotion recognition, existing technologies have not yet achieved highly accurate and reliable identification of emotional states, particularly when recognizing complex or ambiguous emotional expressions. Meanwhile, in the field of neural modulation, current technologies still fall short in real-time monitoring and precise modulation analysis.

[0003] The aforementioned limitations restrict the application of emotion recognition technology and neural modulation systems in daily life, as their accuracy, real-time performance, and reliability are crucial for the precise monitoring of emotion recognition and neural modulation. Summary of the Invention

[0004] To address the aforementioned technical problems, or at least partially address them, this disclosure provides an emotion recognition and neural modulation system and method based on bilateral amygdala phase difference.

[0005] This disclosure provides an emotion recognition and neural modulation system based on bilateral amygdala phase difference, the system comprising:

[0006] Physiological signal acquisition equipment is used to collect users' physiological signals;

[0007] A facial expression capture device that captures the user's facial expressions and eye movement information;

[0008] An electroencephalogram (EEG) device for acquiring the user's brainwave signals;

[0009] Magnetic resonance imaging equipment, used to acquire brain activity maps of the user;

[0010] The data processing module is used to preprocess the physiological signals, facial expressions, and eye movement information and input them into the emotion recognition model to obtain the emotion category. Based on the neural network model, it determines the neural activity pattern according to the electroencephalogram (EEG) signal and the brain activity map. Based on the comparison results of the neural activity pattern and the emotion category, it adjusts the device operating parameters of the emotion recognition model, the EEG device, and the magnetic resonance imaging device.

[0011] This disclosure also provides an emotion recognition and neural modulation method based on bilateral amygdala phase difference, the method comprising:

[0012] Collect users' physiological signals, facial expressions, eye movement information, electroencephalogram signals, and brain activity maps;

[0013] The physiological signals, facial expressions, and eye movement information are preprocessed and then input into the emotion recognition model to obtain the emotion category;

[0014] The neural activity pattern is determined based on the EEG signal and the brain activity map using a neural network model, and the operating parameters of the emotion recognition model, the EEG device, and the MRI device are adjusted based on the comparison results of the neural activity pattern and the emotion category.

[0015] This disclosure also provides an electronic device, the electronic device comprising: a processor; a memory for storing executable instructions of the processor; the processor being configured to read the executable instructions from the memory and execute the instructions to implement the emotion recognition and neural modulation method based on bilateral amygdala phase difference as provided in this disclosure.

[0016] This disclosure also provides a computer-readable storage medium storing a computer program for executing the emotion recognition and neural modulation method based on bilateral amygdala phase difference as provided in this disclosure.

[0017] Compared with existing technologies, the technical solution provided in this disclosure has the following advantages: The emotion recognition and neural modulation scheme based on bilateral amygdala phase difference provided in this disclosure collects the user's physiological signals, facial expressions, eye movement information, electroencephalogram (EEG) signals, and brain activity maps; after preprocessing the physiological signals, facial expressions, and eye movement information, it is input into the emotion recognition model to obtain the emotion category; based on the neural network model, the neural activity pattern is determined according to the EEG signals and brain activity maps, and the operating parameters of the emotion recognition model, EEG equipment, and MRI equipment are adjusted according to the comparison results of the neural activity pattern and the emotion category. By utilizing the synchronous phase difference characteristics of the amygdala, the accuracy and sensitivity of the emotion recognition system are improved, especially for the recognition of complex or ambiguous emotions, thereby improving the accuracy of emotion recognition. Simultaneously, this disclosure also achieves real-time monitoring and precise analysis of the neural modulation system to improve its reliability and practicality in application fields. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0019] Figure 1 A schematic diagram of the structure of an emotion recognition and neural modulation system based on bilateral amygdala phase difference provided in an embodiment of this disclosure;

[0020] Figure 2 This is a flowchart illustrating an emotion recognition and neural modulation method based on bilateral amygdala phase difference, provided in an embodiment of this disclosure. Detailed Implementation

[0021] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0022] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0023] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0024] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0025] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0026] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0027] Emotion recognition technology refers to methods and systems used to identify and analyze human emotional states. Traditional emotion recognition technologies typically rely on physiological signals, facial expressions, or vocal features, as well as questionnaires, to determine an individual's emotions. However, these methods have limitations in terms of complex emotion recognition, real-time performance, and accuracy. Furthermore, existing technologies have not yet provided solutions that meet practical needs for emotion recognition in diverse emotions and complex situations.

[0028] Neuromodulation systems refer to technologies used to monitor, analyze, and regulate the activity of the human nervous system. These systems typically involve the acquisition and analysis of neural signals from the brain, and are used in areas such as the treatment of neurological disorders and mood management. However, existing neuromodulation technologies still face challenges in real-time monitoring, precise analysis, and personalized regulation. In terms of the precise monitoring and regulation of neural activity, current technologies have not yet provided sufficient accuracy and real-time performance.

[0029] Currently, in the field of emotion recognition and neural modulation, there is active exploration of technical solutions based on the constant phase difference of the bilateral amygdala. These solutions include utilizing neuroimaging techniques to focus on the synchronous phase difference characteristics of the amygdala, in order to achieve high-level recognition and precise neural modulation of individual emotional states. Another approach utilizes advanced data processing techniques, combined with machine learning algorithms and pattern recognition technology, to perform real-time monitoring and intelligent analysis of the bilateral amygdala phase difference, in order to gain a more comprehensive understanding and regulation of emotional states.

[0030] Meanwhile, the application of neural network models and deep learning algorithms has become crucial for technological development. These technologies enable intelligent processing and recognition of multi-band neural activity to meet the needs of emotion recognition and neural modulation in complex situations. Furthermore, research on the constant phase difference of the bilateral amygdala has attracted significant attention; this method can achieve personalized nervous system modulation, providing patients with more effective treatment options.

[0031] In-depth research into the phase difference characteristics of the amygdala provides new perspectives on emotion recognition and neural modulation, and offers more accurate information for personalized treatment plans. These innovative methods and technologies open up new possibilities for the treatment of neurological disorders and emotion management.

[0032] This disclosure improves the accuracy and sensitivity of emotion recognition systems by observing and utilizing the synchronous phase difference characteristics of the amygdala, particularly for the recognition of complex or ambiguous emotions, thereby enhancing the accuracy of emotion recognition. Simultaneously, this disclosure also achieves real-time monitoring and precise analysis of neural regulatory systems, improving the reliability and practicality of its applications. Furthermore, this system goes beyond simply combining neuroscience techniques and data processing methods; it opens up new avenues for exploration in the emotion regulation and cognitive processing of the nervous system. By utilizing the constant phase difference between the two amygdalae, high-precision monitoring and regulation of emotional states and neural activity are achieved. This unique approach combines the complexity of neural networks with the correlation between emotion cognition, providing a new perspective for understanding emotion and neurological disorders.

[0033] By capturing and analyzing the synchronous phase difference characteristics of the amygdala, this system achieves more accurate and reliable emotion recognition, providing real-time neural activity monitoring and precise analysis. Combining advanced neuroscience techniques, this system offers a new paradigm for emotion recognition and neural conduction analysis, providing innovative solutions for related fields and striving to improve system reliability and practicality. The innovation behind the system lies not only in its technological integration but also in its new understanding of the interaction between emotion and neural information. It delves into the connection between the phase difference characteristics of the amygdala and emotional cognition, providing new theoretical support for exploring the regulatory mechanisms of emotion formation and neural activity. This phase difference-based approach opens up a new interpretation of the complex network of emotion and cognition, offering new ideas and inspiration for future research in neuroscience and emotion cognition.

[0034] While existing technologies utilize neural network models and machine learning algorithms to process multi-band neural activity, the complexity of data processing can limit real-time performance and efficiency. The computational resources required to process large amounts of real-time data can be substantial, limiting the system's practicality. Furthermore, most existing technologies focus solely on the phase difference characteristics of the bilateral amygdala, neglecting other neural signals related to emotional states. This limitation of relying on a single feature may reduce the system's comprehensiveness and accuracy in emotion recognition and neural modulation, including the fact that it has not fully accounted for individual differences in neural activity. Due to the variability of individual nervous systems, existing technologies may have limitations in providing personalized treatment plans.

[0035] In response to the shortcomings of the existing technologies, the emotion recognition and neural modulation system based on bilateral amygdala phase difference disclosed herein has the following advantages and innovations.

[0036] 1. Network Model Optimization: Three main neural network algorithms (SVM, CNN, and LSTM) were employed to process emotion recognition and neural modulation. These algorithms complement each other, performing comprehensive analysis and processing on different emotional characteristics and neural activities, thereby improving the system's overall understanding and application of emotional and neural information.

[0037] 2. Coupling of Emotion Recognition Network and Neural Regulation System: The output of the emotion recognition network serves as the input of the neural regulation system, establishing a linkage mechanism between the two systems. This coupling enables the neural regulation system to perform real-time neural regulation based on the analysis results of the emotion recognition network, providing individuals with more effective emotion management and regulation solutions.

[0038] 3. Feedback Mechanism Optimization: The output of the neural regulation system serves as feedback, influencing the emotion recognition network and achieving closed-loop feedback. By leveraging the influence of the neural regulation system on the emotion recognition network, the accuracy of emotion recognition and the robustness of the system are further optimized and adjusted, thereby achieving more precise emotion recognition and management.

[0039] 4. Comprehensive Emotional and Neural Information Processing: This approach combines emotion recognition and neural regulation, providing in-depth analysis and processing of the complexity of emotions and neural activity from different perspectives. This comprehensive processing offers more information and feedback, providing more effective tools and methods for medical and personal emotion management.

[0040] These optimizations and innovations will help the emotion recognition and neuromodulation system overcome the limitations of existing technologies, propelling the system to take a more significant step forward in providing more accurate, comprehensive, and personalized emotion recognition and neuromodulation.

[0041] Figure 1 This is a schematic diagram of a structure for an emotion recognition and neural modulation system based on bilateral amygdala phase difference, provided as an embodiment of this disclosure. This system can be implemented by software and / or hardware and is generally integrated into an electronic device. Figure 1 As shown, the system includes:

[0042] Physiological signal acquisition device 101 is used to acquire the user's physiological signals.

[0043] The facial expression acquisition device 102 collects the user's facial expressions and eye movement information.

[0044] Electroencephalography (EEG) device 103 is used to collect the user's brain signals.

[0045] Magnetic resonance imaging device 104 is used to acquire brain activity maps of the user.

[0046] The data processing module 105 is used to preprocess physiological signals, facial expressions and eye movement information and input them into the emotion recognition model to obtain the emotion category. Based on the neural network model, it determines the neural activity pattern according to the EEG signal and brain activity map. Based on the comparison results of the neural activity pattern and the emotion category, it adjusts the equipment operating parameters of the emotion recognition model, the EEG equipment and the MRI equipment.

[0047] In this embodiment of the disclosure, the electroencephalography (EEG) device mainly consists of multichannel brain electrodes, an amplifier, and a recording device. These devices use highly biocompatible materials, such as leather, rubber, and metal, to ensure wearing comfort, provide more stable and accurate data acquisition, and exhibit good sensitivity and reliability in monitoring emotional states.

[0048] In the embodiments disclosed herein, the functional and performance characteristics of the functional magnetic resonance imaging (fMRI) device include high magnetic field strength, fast image acquisition speed, high spatial resolution, and high sensitivity to neural activity, providing high-quality brain imaging capabilities, providing reliable data support for studies on emotion recognition and neural modulation, accurately recording brain functional connectivity, and providing detailed brain activity maps for exploring the mechanisms of emotion formation and neural activity.

[0049] In addition to high-quality imaging capabilities, the software and data analysis tools of fMRI equipment were also considered. These devices typically come equipped with advanced imaging processing software, such as the Siemens syngo MR E11 platform, for data acquisition, image reconstruction, and analysis.

[0050] In one embodiment of this disclosure, the data processing module 105 is further configured to: acquire the target EEG signal and target brain activity map after the EEG device and the magnetic resonance imaging device have adjusted their operating parameters.

[0051] In one embodiment of this disclosure, the physiological signal acquisition device 101 includes: a heart rate monitor and a skin conductance meter; the heart rate monitor is used to acquire the user's heart rate information; the skin conductance meter is used to acquire the user's skin conductance response and psychological stress information.

[0052] In this embodiment, the heart rate monitor can effectively record heart rate changes and has a high-precision heart rate monitoring function. It is not only stable and reliable but also provides accurate heart rate data, which helps to comprehensively understand the relationship between emotional changes and heart rate.

[0053] In this embodiment of the disclosure, the skin conductance meter can record skin conductance responses and psychological stress, providing monitoring of the activity of the human autonomic nervous system. It performs well in recording skin conductance responses and emotional changes, providing a more comprehensive and in-depth understanding of emotional states.

[0054] In the embodiments of this disclosure, eye trackers can accurately record the movement trajectory and focus of a person's eyes, helping to better understand an individual's attention allocation and emotional expression. They possess high-precision eye-tracking capabilities, providing important auxiliary information for research on emotion recognition.

[0055] In the embodiments of this disclosure, facial expression recognition cameras are capable of capturing and recognizing facial expressions, and analyzing and recognizing the expression of emotions. They possess advanced technology and efficient facial expression analysis capabilities in the field of facial expression recognition.

[0056] In one embodiment of this disclosure, the data processing module 105 is further configured to: acquire physiological signal samples, facial expression samples, and eye movement information samples; extract and fuse features from the physiological signal samples, facial expression samples, and eye movement information samples respectively to obtain bilateral amygdala constant phase difference fusion features; and train the network model based on the support vector machine algorithm using the bilateral amygdala constant phase difference fusion features as a self-supervised set to obtain an emotion recognition model.

[0057] In one embodiment of this disclosure, the data processing module 105 is specifically used to: extract features and process spatial information from EEG signals and brain activity maps based on a convolutional neural network, and process the EEG signals through a long short-term memory network to obtain the temporal correlation between signals from different neurons and determine neural activity patterns.

[0058] In one embodiment of this disclosure, the data processing module 105 employs data processing software such as MATLAB to process physiological signals and neuroimaging data, perform feature extraction and pattern recognition, and also conduct machine learning and data modeling. Its powerful data processing and analysis capabilities, along with its rich toolbox, make it one of the preferred tools in the field of scientific research. Additionally, Python and its related libraries are also used. Python has wide applicability, and its powerful data processing capabilities and rich library support make it a popular choice in fields such as deep learning and machine learning for developing and implementing various complex neural network models.

[0059] Additionally, computer servers are selected to handle large-scale data and model training. Furthermore, GPUs are used, for example, to accelerate the CUDA deep learning framework, speeding up model training and optimization. These hardware devices are characterized by providing high-performance computing and processing capabilities, significantly improving the system's real-time performance and efficiency to meet the demands of large-scale data processing and model training.

[0060] In one embodiment of this disclosure, the emotion recognition and neural modulation system based on bilateral amygdala phase difference further includes: a human-computer interaction module for displaying emotion categories, target EEG signals, and target brain activity maps in a target format.

[0061] In one embodiment of this disclosure, the human-computer interaction module is further configured to: encrypt the emotion category, the target EEG signal, and the target brain activity map according to a preset method and upload them to a cloud server.

[0062] In this embodiment, the human-computer interaction module, i.e., the user interface and feedback system, employs a custom-developed application or mobile application platform to provide a user-friendly interface and real-time emotional feedback. Simultaneously, visualization tools such as D3.js and Plotly are used for data visualization, providing users with an intuitive data display and feedback mechanism.

[0063] In this embodiment, physiological signal samples, facial expression samples, and eye movement information samples are acquired. Features are extracted and fused from the physiological signal samples, facial expression samples, and eye movement information samples respectively to obtain bilateral amygdala constant phase difference fusion features. Based on the support vector machine algorithm, the network model is trained using the bilateral amygdala constant phase difference fusion features as a self-supervised set to obtain an emotion recognition model.

[0064] This module trains an SVM model, i.e., an emotion recognition model, using a large number of constant phase differences between the two-sided amygdala as features in the training data. This model can learn and analyze these features to classify new input data based on their relationship with emotional states. Specifically, this module uses SVM to determine the association between a given feature vector and a known emotional state, and infers the possible emotion category accordingly. The following is the classifier function expression for the SVM: f(x) represents the emotional state predicted from input data x (such as physiological signals, neural data, etc.). N is the number of emotional samples in the training set. α i This indicates the importance of different features or physiological signals used for emotion classification. γ i These are category labels, representing different categories of emotions, such as happiness, sadness, etc. K(x,x) i () is a kernel function used to measure the difference between the input data x and the training samples x. i The similarity between them. b represents the threshold or offset of the emotion classifier.

[0065] When integrated into an emotion recognition and neural modulation system, the SVM module will serve as a key component, using bilateral amygdala constant phase difference data for training, thereby achieving accurate classification and recognition of different emotional states.

[0066] In this embodiment of the disclosure, feature extraction and spatial information processing are performed on the electroencephalogram (EEG) signals and the brain activity map based on a convolutional neural network, and the EEG signals are processed through a long short-term memory (LSTM) network to obtain the temporal correlation between signals from different neurons and determine the neural activity pattern.

[0067] The Convolutional Neural Network (CNN) consists of the following layers: Input: Pre-processed neurophysiological signal data (removed power frequency interference, rereferenced, depolarized, discrete Fourier transform, interpolated reconstruction); Convolutional layers: Extract physiological features and detect signal patterns from the input neural signal data; Pooling layers: Reduce data dimensionality, retain important features, remove irrelevant information and noise, reduce the complexity of neural signals, and highlight key neural regulation patterns; ReLU activation function: Increase the nonlinear features of the network and enhance its semantic expressive ability; Fully connected layers: Integrate and classify the neural signal features extracted by the convolutional layers; Deactivation layers: Reduce overfitting by randomly deactivating some neurons to reduce the interdependence between neurons in the neural network and increase the network's generalization ability; Batch normalization layers: Added after the fully connected layers to help improve training speed and network stability; Output: For neural regulation systems, the output layer includes the classification of emotional states and neural activities.

[0068] Specifically, convolutional neural networks (CNNs) can help analyze and understand neural signals and physiological data in neural regulation systems, thus providing useful information for neural regulation. Through the above network architecture and reasonable and appropriate training, CNNs can extract regulatory patterns from complex neural data and help identify and analyze specific neural activities or effects.

[0069] Specifically, Long Short-Term Memory (LSTM) networks are a special type of Recurrent Neural Network (RNN) that addresses the long-term dependency problem of traditional RNNs, enabling them to better handle time-series neural signal data. Through its gating structure, LSTM can selectively ignore or store neural information, while accessing and retaining important analytical neural signals when needed.

[0070] In this embodiment, the LSTM network has a flexible architecture capable of adapting to sequence data of varying lengths and automatically learning and adjusting its internal parameters as needed to suit different neural modulation tasks. The neural modulation system employs a multi-layered stacked LSTM architecture. By stacking multiple LSTM layers, the system can progressively extract and organize complex time-series features, improving the model's representational power and accuracy.

[0071] In addition, the LSTM network used in the neural regulation system is also suitable for processing a variety of physiological information, including physiological signals, heart rate changes, etc., and can be applied to real-time monitoring and analysis of neural activity, thereby assisting in emotion recognition and emotion management.

[0072] Understandably, the emotion recognition and neural modulation system based on bilateral amygdala phase difference consists of two parts: an emotion recognition network and a neural modulation system. The output of the emotion recognition network simultaneously serves as the input or reference for the neural modulation system. The emotion recognition network model can provide an individual's current emotional state, and the neural modulation system can adjust its neural activity accordingly to help improve emotions or regulate the physiological responses involved in emotions. This combined use helps to better understand the relationship between emotions and neural activity, thereby providing more precise neural modulation strategies.

[0073] Specifically, the entire system utilizes two main modules: emotion recognition and neural regulation. These modules enable the identification of emotional data, monitoring of neural activity, and the regulation and feedback of individual emotional states, providing users with comprehensive information to support more accurate analysis and decision-making. The following is a detailed description of the process.

[0074] Specifically, data acquisition includes the collection of physiological signals, facial expressions, and eye movements. Heart rate monitors and electrodermal response (EDR) devices are used to record individual physiological signal data. Heart rate monitors record heart rate changes, while EDR devices record skin conductance responses and psychological stress levels. Eye trackers and facial expression recognition cameras are used to capture and identify individual facial expressions and eye movement information. These devices assist in emotion recognition, providing important data for emotion analysis.

[0075] Specifically, data preprocessing involves performing preliminary preprocessing on the various data collected through the above methods, including filtering, noise reduction, and feature extraction, which then constitutes the input part of the data processing module.

[0076] Specifically, the data processing module includes the construction of an emotion recognition network: through feature fusion and model training of the algorithm network, features from physiological signals and facial expressions / eye movements are fused to establish a comprehensive emotion recognition model. The SVM (Support Vector Machines) algorithm is used to train the network model using the fused features as a self-supervised set to achieve accurate identification of emotional states. The neural regulation system is constructed by combining the emotion recognition model with real-time neural activity monitoring, using a CNN as the core network architecture and LSTM as a key component to build the neural regulation system. That is, pre-processed neural signal data and physiological signal data, such as those processed through power line interference removal, rereference, depolarization, discrete Fourier transform, and interpolation reconstruction, are input into the CNN. The CNN is used for neural feature extraction and spatial information processing, while the LSTM is responsible for integrating temporal information sequences, analyzing and understanding the temporal characteristics of neural signal data, and capturing the temporal correlation between signals from different neurons. This system will accurately output data that reflects the individual's neural activity state, thereby achieving the prediction and regulation of the individual's neural activity.

[0077] Specifically, in processing emotion recognition results and neural activity monitoring, neural modulation systems typically involve two main aspects: using emotion recognition results to adjust the frequency and focus of neural activity monitoring, and using real-time neural activity monitoring data to validate and optimize the accuracy of the emotion recognition model.

[0078] As an example, when an emotion recognition network determines that an individual is in a state of happiness, the neuromodulation system can adjust the frequency and focus of neural activity monitoring. In a happy state, it may focus more on specific activities in certain brain regions, or increase the monitoring frequency to gain a more detailed understanding of the dynamic changes in the individual's neural activity. The neuromodulation system adjusts its monitoring strategy in real time based on these changes to better adapt to the individual's current emotional state. This can be achieved by adjusting the sampling frequency of the monitoring device, selecting to monitor activity in specific brain regions, or increasing attention to specific neural signals.

[0079] As another example, by analyzing neural activity data, the system can identify specific neural activity patterns under different emotional states and compare these patterns with the output of an emotion recognition network. If the neural activity monitoring data is inconsistent with the emotion recognition results, the system can provide feedback to optimize the emotion recognition model. This might include retraining the emotion recognition network, adjusting the weights of feature fusion, or updating the algorithm to better capture an individual's emotional state.

[0080] In this embodiment, a custom application or mobile application platform is developed, and real-time network output data is uploaded to the platform. A user-friendly interface and real-time emotional feedback are then provided to doctors through visualization tools. Doctors can obtain real-time emotional data and analysis results through the application, enabling them to monitor and regulate patients' emotional states.

[0081] In this embodiment, the results of neuromodulation are presented through a doctor's interface. The patient's emotional state, neuromodulation results, and corresponding analysis are displayed in target formats such as charts and data reports to aid doctors in making clinical diagnoses and decisions.

[0082] In this embodiment of the disclosure, doctors can download various data from this process to a dedicated cloud database, using encryption and access control measures to protect the patient's sensitive data, while also providing support for future system optimization and adjustments.

[0083] Therefore, the emotion recognition and neural modulation system based on bilateral amygdala phase difference provided in this disclosure improves the accuracy and sensitivity of the emotion recognition system by observing and utilizing the synchronous phase difference characteristics of the amygdala, thereby improving the accuracy of emotion recognition. Simultaneously, it also enables real-time monitoring and precise analysis of the neural modulation system, thereby enhancing its reliability and practicality in application fields.

[0084] Specifically, Figure 2 This is a flowchart illustrating an emotion recognition and neural modulation method based on bilateral amygdala phase difference, provided in an embodiment of this disclosure. This method can be executed by an emotion recognition and neural modulation device based on bilateral amygdala phase difference, which can be implemented in software and / or hardware and is generally integrated into an electronic device. Figure 2 As shown, the method includes:

[0085] Step 201: Collect the user's physiological signals, facial expressions, eye movement information, electroencephalogram (EEG) signals, and brain activity maps.

[0086] Step 202: After preprocessing the physiological signals, facial expressions, and eye movement information, input them into the emotion recognition model to obtain the emotion category.

[0087] Step 203: Based on the neural network model, determine the neural activity pattern according to the EEG signal and the brain activity map, and adjust the operating parameters of the emotion recognition model, EEG equipment and magnetic resonance imaging equipment according to the comparison results of neural activity pattern and emotion category.

[0088] In this embodiment, target EEG signals and target brain activity maps are acquired after adjusting the operating parameters of an EEG and a magnetic resonance imaging (MRI) device. The emotion category, target EEG signal, and target brain activity map are displayed in a target format. The emotion category, target EEG signal, and target brain activity map are encrypted according to a preset method and uploaded to a cloud server.

[0089] Specifically, in processing emotion recognition results and neural activity monitoring, neural modulation systems typically involve two main aspects: using emotion recognition results to adjust the frequency and focus of neural activity monitoring, and using real-time neural activity monitoring data to validate and optimize the accuracy of the emotion recognition model.

[0090] As an example, when an emotion recognition network determines that an individual is in a state of happiness, the neuromodulation system can adjust the frequency and focus of neural activity monitoring. In a happy state, it may focus more on specific activities in certain brain regions, or increase the monitoring frequency to gain a more detailed understanding of the dynamic changes in the individual's neural activity. The neuromodulation system adjusts its monitoring strategy in real time based on these changes to better adapt to the individual's current emotional state. This can be achieved by adjusting the sampling frequency of the monitoring device, selecting to monitor activity in specific brain regions, or increasing attention to specific neural signals.

[0091] As another example, by analyzing neural activity data, the system can identify specific neural activity patterns under different emotional states and compare these patterns with the output of an emotion recognition network. If the neural activity monitoring data is inconsistent with the emotion recognition results, the system can provide feedback to optimize the emotion recognition model. This might include retraining the emotion recognition network, adjusting the weights of feature fusion, or updating the algorithm to better capture an individual's emotional state.

[0092] The emotion recognition and neural modulation method based on bilateral amygdala phase difference provided in this disclosure can execute the emotion recognition and neural modulation system based on bilateral amygdala phase difference provided in any embodiment of this disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0093] This disclosure also provides a computer program product, including a computer program / instruction that, when executed by a processor, implements the emotion recognition and neural modulation method based on bilateral amygdala phase difference provided in any embodiment of this disclosure.

[0094] According to one or more embodiments of this disclosure, this disclosure provides an electronic device, including:

[0095] processor;

[0096] Memory used to store the processor's executable instructions;

[0097] The processor is configured to read the executable instructions from the memory and execute the instructions to implement the emotion recognition and neural modulation method based on bilateral amygdala phase difference as described in any of the present disclosure.

[0098] According to one or more embodiments of the present disclosure, the present disclosure provides a computer-readable storage medium storing a computer program for performing an emotion recognition and neural modulation method based on bilateral amygdala phase difference as described in any of the present disclosure.

[0099] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0100] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0101] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. An emotion recognition and neural modulation system based on bilateral amygdala phase difference, characterized in that, include: Physiological signal acquisition equipment is used to collect users' physiological signals; A facial expression capture device that captures the user's facial expressions and eye movement information; An electroencephalogram (EEG) device for acquiring the user's brainwave signals; Magnetic resonance imaging equipment, used to acquire brain activity maps of the user; The data processing module is used to preprocess the physiological signals, facial expressions, and eye movement information and input them into the emotion recognition model to obtain the emotion category. Based on the neural network model, it determines the neural activity pattern according to the electroencephalogram (EEG) signal and the brain activity map. Based on the comparison results of the neural activity pattern and the emotion category, it adjusts the device operating parameters of the emotion recognition model, the EEG device, and the magnetic resonance imaging device.

2. The emotion recognition and neural modulation system based on bilateral amygdala phase difference according to claim 1, characterized in that, The data processing module is also used for: The target EEG signal and target brain activity map are obtained after the operating parameters of the EEG device and the MRI device are adjusted.

3. The emotion recognition and neural modulation system based on bilateral amygdala phase difference according to claim 1, characterized in that, The physiological signal acquisition device includes: a heart rate monitor and a skin conductance meter; The heart rate monitor is used to collect the user's heart rate information; The skin conductance meter is used to collect the user's skin conductance response and psychological stress information.

4. The emotion recognition and neural modulation system based on bilateral amygdala phase difference according to claim 1, characterized in that, The data processing module is also used for: Acquire physiological signal samples, facial expression samples, and eye movement information samples; Feature extraction and fusion were performed on the physiological signal samples, facial expression samples, and eye movement information samples respectively to obtain bilateral amygdala constant phase difference fusion features; The emotion recognition model is obtained by training the network model based on the support vector machine algorithm and using the constant phase difference fusion features of the two-sided amygdala as a self-supervised set.

5. The emotion recognition and neural modulation system based on bilateral amygdala phase difference according to claim 1, characterized in that, The data processing module is specifically used for: Feature extraction and spatial information processing are performed on the EEG signals and brain activity maps based on convolutional neural networks, and the EEG signals are processed through long short-term memory networks to obtain the temporal correlation between signals of different neurons and determine the neural activity patterns.

6. The emotion recognition and neural modulation system based on bilateral amygdala phase difference according to claim 2, characterized in that, Also includes: The human-computer interaction module is used to display the emotion category, the target EEG signal, and the target brain activity map in a target format.

7. The emotion recognition and neural modulation system based on bilateral amygdala phase difference according to claim 6, characterized in that, The human-computer interaction module is also used for: The emotion category, the target EEG signal, and the target brain activity map are encrypted according to a preset method and uploaded to a cloud server.

8. A method for emotion recognition and neural modulation based on bilateral amygdala phase difference, characterized in that, include: Collect users' physiological signals, facial expressions, eye movement information, electroencephalogram signals, and brain activity maps; The physiological signals, facial expressions, and eye movement information are preprocessed and then input into the emotion recognition model to obtain the emotion category; The neural activity pattern is determined based on the EEG signal and the brain activity map using a neural network model, and the operating parameters of the emotion recognition model, EEG device and MRI device are adjusted based on the comparison results of the neural activity pattern and the emotion category.

9. An electronic device, characterized in that, The electronic device includes: processor; Memory used to store the processor's executable instructions; The processor is configured to read the executable instructions from the memory and execute the instructions to implement the emotion recognition and neural modulation method based on bilateral amygdala phase difference as described in claim 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program for executing the emotion recognition and neural modulation method based on bilateral amygdala phase difference as described in claim 8.

Images