Multi-mode emotion feedback regulation system

Abstract

The invention relates to the technical field of emotion recognition and regulation, and discloses a multi-modal emotion feedback regulation system, which comprises the following modules: an emotion recognition module used for acquiring an emotion state of a user through multi-modal data, the multi-modal data including but not limited to physiological signals, voice signals, facial expressions, action recognition and text analysis results; and the emotion analysis and modeling module is used for performing analysis and modeling on the emotion state of the user through a deep learning model according to the acquired multi-modal data, and generating time sequence data of the emotion of the user. By integrating a multi-mode emotion recognition technology and combining various data sources such as physiological signals, voice, facial expressions, actions and text data, the emotion state of the user can be comprehensively and accurately evaluated. Compared with traditional single-mode recognition, the multi-dimensional emotion recognition method has higher accuracy and adaptability, so that emotion regulation feedback can provide personalized feedback strategies more accurately according to individual requirements of users.

Description

Technical Field

[0001] The present invention relates to the technical field of emotion recognition and regulation, and in particular to a multimodal emotion feedback regulation system. Background Art

[0002] With the acceleration of social pace and the increase of pressure, emotion management has become an important issue in people's daily life and work. Emotion regulation not only affects the mental health of individuals, but also largely determines their interpersonal relationships, work efficiency and quality of life. Therefore, how to help users regulate their emotions through scientific and effective means has become a hot topic in many fields such as psychology, artificial intelligence, and human biology.

[0003] Existing emotion recognition and regulation technologies can be roughly divided into two categories: one is emotion assessment and regulation based on single-modal data, such as facial expression recognition, voice emotion analysis, or physiological signal monitoring. These technologies infer the user's emotional state and give certain feedback by analyzing a single data source (such as facial expressions or voice). However, since emotions are multidimensional and complex, single-modal data often cannot fully and accurately reflect an individual's emotional fluctuations. For example, the emotional expression of voice is affected by factors such as the speaking environment and language expression, while facial expressions are also biased due to cultural differences or individual habits. Therefore, single-modal emotion recognition methods often face limitations such as low accuracy and poor adaptability.

[0004] The other type is a method that combines multimodal data for emotion regulation, but the existing multimodal emotion recognition and regulation systems still have many shortcomings. For example, although some systems are able to integrate multiple data sources such as vision, voice and physiological signals for emotion recognition, their feedback mechanisms are often too simple and lack real-time dynamic adjustment of the user's emotional state. Most systems only give fixed regulation feedback based on preliminary emotion assessment results, and are unable to optimize and adjust according to changes in user emotions, resulting in unsatisfactory feedback effects. In addition, the emotion regulation strategies in the existing technology are often extensive, lack personalized and detailed regulation methods, and cannot be flexibly adapted to the needs of different users.

[0005] To this end, those skilled in the art have proposed a multimodal emotion feedback regulation system to solve the above problems. Summary of the invention

[0006] In view of the deficiencies of the prior art, the present invention provides a multimodal emotion feedback regulation system, which solves the problems raised in the above background technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions: a multimodal emotion feedback regulation system, comprising the following modules:

[0008] An emotion recognition module, used to obtain the user's emotional state through multimodal data, the multimodal data including but not limited to physiological signals, voice signals, facial expressions, action recognition and text analysis results;

[0009] The emotion analysis and modeling module is used to analyze and model the user's emotional state through a deep learning model based on the acquired multimodal data, and generate time series data of the user's emotions;

[0010] An emotion regulation module is used to provide targeted emotion regulation feedback based on the user's emotional state, and the feedback methods include but are not limited to voice feedback, visual feedback, tactile feedback or environmental adjustment;

[0011] Feedback optimization module, which is used to optimize the emotion regulation feedback strategy through reinforcement learning algorithm to maximize the positive change of the user's emotional state;

[0012] The user interface module is used to display the user's real-time emotional state and regulation effect in a visual way, and provide personalized emotion regulation suggestions.

[0013] Preferably, the emotion recognition module includes the following submodules:

[0014] A physiological signal acquisition unit, used to collect the user's physiological data through sensors, the physiological data including but not limited to heart rate, skin electrical response, electroencephalogram signal, etc.;

[0015] A speech signal analysis unit, used to perform emotion classification on the user's speech signal through speech recognition and speech emotion analysis models, and extract emotional features from the speech;

[0016] Facial expression recognition unit, used to identify the emotional changes on the user's face, such as happiness, anger, sadness, etc., through computer vision technology and facial recognition algorithms;

[0017] The motion recognition unit is used to capture the user's motion and posture through sensors or cameras, and recognize the user's emotional fluctuations based on the motion data.

[0018] Preferably, the sentiment analysis and modeling module adopts a fusion model to jointly analyze multiple modal data, specifically through the following steps:

[0019] Data preprocessing: standardizing the data of each modality, including but not limited to normalization and missing value filling;

[0020] The emotion classification model uses deep learning methods such as convolutional neural networks and long short-term memory networks to perform emotion recognition based on processed multimodal data and output emotion categories;

[0021] Emotion intensity assessment, based on the fusion of modal data, predicts the emotion intensity through a regression model, where the emotion intensity prediction formula is as follows:

[0022] Emotional intensity = α1HR+α2EDA+α3 speech emotion features

[0023] +α4 facial expression features +α5 action features

[0024] Among them, α1, α2, α3, α4, and α5 are the weight coefficients of the model.

[0025] Preferably, the emotion regulation module provides a variety of adjustment feedback methods based on the user's real-time emotional state, including:

[0026] The voice feedback unit generates voice feedback with emotion regulation effect through speech synthesis technology according to the user's emotional state. The content and tone of the voice feedback are personalized according to the user's emotional characteristics;

[0027] The visual feedback unit presents different colors, graphics or animations through display screens, LED lights and other devices according to the user's emotional state to adjust the user's emotions;

[0028] The tactile feedback unit provides tactile feedback to users through vibration, temperature change, etc. to help users adjust their emotions;

[0029] The environmental adjustment unit improves the user's emotional state by adjusting the environment around him.

[0030] Preferably, the feedback optimization module optimizes the feedback strategy based on a reinforcement learning algorithm, specifically:

[0031] Reward function, which defines the reward function of the feedback strategy, gives rewards based on changes in the user's emotional state. The formula is as follows:

[0032] R=ΔEmotional intensity·w1+ΔEmotional stability·w1

[0033] Among them, Δemotional intensity is the change in the user's emotional intensity, Δemotional stability is the amplitude of emotional fluctuation, and w1 and w2 are weight coefficients.

[0034] Preferably, the user interface module is used to display the user's real-time emotional state and adjustment effect, including:

[0035] Real-time emotional state visualization unit, which displays the user's current emotional state through a dynamic dashboard, including information such as emotional category and emotional intensity;

[0036] The regulation effect display unit displays the effect of feedback regulation in real time, such as the change curve of emotion intensity, the change of emotion stability, etc.

[0037] The emotion regulation suggestion generation unit provides personalized emotion regulation suggestions based on the user's emotional fluctuations and historical data, including recommended regulation methods, feedback types and frequencies, etc.

[0038] Preferably, the system further includes an emotion prediction module for predicting future emotion change trends based on the user's historical emotion data, specifically:

[0039] The emotional trend prediction model predicts future emotional change trends based on the user's historical emotional data through a time series prediction algorithm. The formula used by the emotional trend prediction model is as follows:

[0040] E t+1 =f(E t ,X t ,θ)

[0041] Where: E t is the emotional intensity at time t, X t is the environment variable at time t,

[0042] θ is the model parameter and f is the emotion prediction function.

[0043] Preferably, the system has an automatic learning function and can continuously adjust the regulation strategy based on user feedback and historical data.

[0044] Preferably, the emotion feedback system can adjust the feedback strategy in real time when receiving external stimulation.

[0045] Preferably, the system can be connected to other devices or systems through a cloud platform to obtain external data or work in collaboration with other emotion regulation systems.

[0046] The present invention provides a multi-modal emotion feedback regulation system, which has the following beneficial effects:

[0047] 1. The present invention integrates multimodal emotion recognition technology and combines multiple data sources such as physiological signals, voice, facial expressions, movements and text data to comprehensively and accurately assess the user's emotional state. This multi-dimensional emotion recognition method has higher accuracy and adaptability than traditional single-modality recognition, so that emotion regulation feedback can more accurately provide personalized feedback strategies for the individual needs of users. Through multiple feedback methods such as voice, vision, and touch, users can get the most suitable adjustment method under different emotional states, thereby significantly improving the adjustment effect and effectively helping users maintain emotional balance and stability.

[0048] 2. The present invention uses a reinforcement learning algorithm to continuously optimize the emotion regulation strategy, and dynamically adjusts it based on the changes in the user's real-time emotional state to maximize the positive changes in emotions. Through the reward function, the system can update the feedback strategy after each interaction, thereby continuously improving the regulation effect. The user's emotional feedback is continuously learned and adapted through multiple interactions, and the feedback mechanism is continuously optimized, so that the system can more accurately identify the user's emotional fluctuations and take effective regulation measures, thereby significantly improving the long-term effect and user satisfaction of the emotion regulation system.

[0049] 3. The present invention combines an emotional trend prediction module to analyze the user's historical emotional data, predict future emotional change trends, and prepare and adjust feedback strategies in advance. The prediction model uses an advanced time series prediction algorithm to capture the laws of user emotional changes, predict emotional fluctuations in the short term, and make corresponding emotional regulation interventions in advance. Combined with the system's automatic learning function, the system can continuously optimize the regulation strategy based on the continuously accumulated user feedback and historical data, so that the system can still respond flexibly and accurately when facing different situations and environmental changes, thereby improving the adaptability and effectiveness of overall emotional regulation. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] Figure 1 is the emotion recognition flow chart of the present invention;

[0051] Figure 2 is a flow chart of sentiment analysis and modeling of the present invention;

[0052] Figure 3 is the emotion regulation flow chart of the present invention;

[0053] Figure 4 A flow chart is shown for the user interface of the present invention. DETAILED DESCRIPTION

[0054] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] Please see attached Figure 1 - Attachment Figure 4 , an embodiment of the present invention provides a multimodal emotion feedback regulation system, comprising the following modules:

[0056] An emotion recognition module, used to obtain the user's emotional state through multimodal data, the multimodal data including but not limited to physiological signals, voice signals, facial expressions, action recognition and text analysis results;

[0057] The emotion recognition module includes the following sub-modules:

[0058] A physiological signal acquisition unit, used to collect the user's physiological data through sensors, the physiological data including but not limited to heart rate, skin electrical response, electroencephalogram signal, etc.;

[0059] Specifically, the physiological signal acquisition unit collects the user's physiological data in real time through various sensor devices (such as wearable sensors, smart bracelets, brain wave acquisition devices, etc.). These data include but are not limited to:

[0060] Heart rate: Heart rate changes are closely related to emotional states (such as anxiety, anger, calmness, etc.). Through changes in heart rate, the user's emotional fluctuations can be monitored in real time.

[0061] Galvanic skin response: Galvanic skin response is the physiological manifestation of emotional response. Especially under emotional stimulation (such as stress, anxiety, etc.), the skin conductivity will change.

[0062] Electroencephalogram: Electroencephalogram signals provide real-time feedback of brain activity, and by analyzing brain wave patterns, it is possible to further identify the user’s emotional state. For example, higher alpha wave frequencies are associated with relaxation, while higher beta wave frequencies are associated with tension or alertness.

[0063] A speech signal analysis unit, used to perform emotion classification on the user's speech signal through speech recognition and speech emotion analysis models, and extract emotional features from the speech;

[0064] Specifically, the voice signal analysis unit uses voice recognition technology and sentiment analysis algorithms to process the user's voice data and perform emotion recognition and feature extraction. Its main functions include:

[0065] Speech recognition: Identify the content information in speech by processing and decoding speech signals.

[0066] Voice sentiment analysis: Identify the user's emotions when speaking by analyzing voice characteristics such as pitch, rhythm, volume, pauses, etc. For example, a higher pitch and a fast speaking rate indicates anger, while a lower pitch and a slow speaking rate indicates depression or sadness.

[0067] Facial expression recognition unit, used to identify the emotional changes on the user's face, such as happiness, anger, sadness, etc., through computer vision technology and facial recognition algorithms;

[0068] Specifically, the facial expression recognition unit is based on computer vision technology, which uses a camera to capture the changes in the user's face in real time and identify related emotional features. The unit performs emotion recognition in the following ways:

[0069] Facial landmark detection: Detect the movement and changes of facial landmarks.

[0070] Facial expression classification: Based on the deep learning model, it analyzes the changes in facial expressions and classifies them into common emotions such as happiness, anger, surprise, sadness, disgust, fear, etc.

[0071] Emotion intensity assessment: It can not only identify the type of emotion, but also assess its intensity, such as the user's smile intensity, anger level, etc.

[0072] The motion recognition unit is used to capture the user's motion and posture through sensors or cameras, and recognize the user's emotional fluctuations based on the motion data.

[0073] The emotion analysis and modeling module is used to analyze and model the user's emotional state through a deep learning model based on the acquired multimodal data, and generate time series data of the user's emotions;

[0074] Specifically, the emotion analysis and modeling module is the core part of the system, responsible for in-depth analysis and modeling of the collected multimodal data. Through deep learning technology, this module can combine various emotion data sources to accurately build a user's emotional state model and generate a corresponding emotion change time series.

[0075] Data fusion: This module can fuse multimodal data from physiological signals, voice, facial expressions, action recognition, etc., and extract emotional features from each data type.

[0076] Emotional modeling: Through deep neural networks or other machine learning algorithms, different types of emotional data are modeled to build a user emotional state model with temporal and complexity. This model can not only capture current emotions, but also predict the user's future emotional trends.

[0077] Prediction of emotional changes: Based on time series data analysis, predict the changing trend of user emotions, thereby providing real-time feedback adjustment basis for the system.

[0078] The sentiment analysis and modeling module uses a fusion model to jointly analyze multiple modal data, specifically through the following steps:

[0079] Data preprocessing: standardizing the data of each modality, including but not limited to normalization and missing value filling;

[0080] Specifically, the pre-processing steps include but are not limited to:

[0081] Normalization: Normalize data of different modalities so that they can be compared at the same scale. For example, standardize the amplitude of audio signals, the numerical range of facial expression features, and the word vectors of text to prevent certain modalities from dominating the model training due to their numerical range being too large or too small.

[0082] Missing value filling: Some modes in multimodal data may have missing values, for example, missing facial expressions in a certain frame in a video stream, or missing some pronunciation features in speech recognition. Common missing value filling methods include mean filling, interpolation, similarity-based filling, etc.

[0083] Feature extraction: For each modality of data, the data is converted into a numerical form that can be processed by the model through a suitable feature extraction method. For example, audio can extract Mel frequency cepstral coefficients, facial expressions can extract key point positions, and text can be represented by word vectors or sentiment dictionaries.

[0084] The emotion classification model uses deep learning methods such as convolutional neural networks and long short-term memory networks to perform emotion recognition based on processed multimodal data and output emotion categories;

[0085] The core ideas of specific emotion classification include:

[0086] Convolutional Neural Network: It is used to process images, videos, and data with spatial structures. It can automatically extract spatial features from input data and is very effective for facial expression images, visual signals, and even some audio signals (such as spectrograms). Through convolutional layers and pooling layers, convolutional neural networks can capture local emotional features and finally perform emotion classification through fully connected layers.

[0087] Long Short-Term Memory Network: Long Short-Term Memory Network is a special recursive neural network that can effectively process time series data. For processing speech, text or time series data (such as the dynamic changes of body language), it can remember and capture the time dependency of emotional information. For example, changes in pitch in speech signals and changes in emotional vocabulary in text can all be analyzed through this network.

[0088] Emotion intensity assessment, based on the fusion of modal data, predicts the emotion intensity through a regression model, where the emotion intensity prediction formula is as follows:

[0089] Emotional intensity = α1HR+α2EDA+α3 speech emotion features

[0090] +α4 facial expression features +α5 action features

[0091] Among them, α1, α2, α3, α4, and α5 are the weight coefficients of the model.

[0092] An emotion regulation module is used to provide targeted emotion regulation feedback based on the user's emotional state, and the feedback methods include but are not limited to voice feedback, visual feedback, tactile feedback or environmental adjustment;

[0093] The emotion regulation module provides multiple adjustment feedback methods based on the user's real-time emotional state, including:

[0094] The voice feedback unit generates voice feedback with emotion regulation effect through speech synthesis technology according to the user's emotional state. The content and tone of the voice feedback are personalized according to the user's emotional characteristics;

[0095] The visual feedback unit presents different colors, graphics or animations through display screens, LED lights and other devices according to the user's emotional state to adjust the user's emotions;

[0096] The tactile feedback unit provides tactile feedback to users through vibration, temperature change, etc. to help users adjust their emotions;

[0097] The environmental adjustment unit improves the user's emotional state by adjusting the environment around him.

[0098] Feedback optimization module, which is used to optimize the emotion regulation feedback strategy through reinforcement learning algorithm to maximize the positive change of the user's emotional state;

[0099] The feedback optimization module optimizes the feedback strategy based on the reinforcement learning algorithm, specifically:

[0100] Reward function, which defines the reward function of the feedback strategy, gives rewards based on changes in the user's emotional state. The formula is as follows:

[0101] R=ΔEmotional Intensity·w1+ΔEmotional Stability·w2

[0102] Among them, Δemotional intensity is the change in the user's emotional intensity, Δemotional stability is the amplitude of emotional fluctuation, and w1 and w2 are weight coefficients.

[0103] The user interface module is used to display the user's real-time emotional state and regulation effect in a visual way, and provide personalized emotion regulation suggestions.

[0104] The system further includes an emotion prediction module, which is used to predict future emotion change trends based on the user's historical emotion data, specifically:

[0105] The emotional trend prediction model predicts future emotional change trends based on the user's historical emotional data through a time series prediction algorithm. The formula used by the emotional trend prediction model is as follows:

[0106] E t+1 =f(E t ,Xt ,θ)

[0107] Where: E t is the emotional intensity at time t, X t is the environmental variable at time t, θ is the model parameter, and f is the emotion prediction function. The system has an automatic learning function and can continuously adjust the adjustment strategy based on user feedback and historical data.

[0108] When receiving external stimuli, the emotion feedback system can adjust the feedback strategy in real time. The system can connect to other devices or systems through the cloud platform to obtain external data or work together with other emotion regulation systems.

[0109] Specifically, the emotion feedback regulation system mainly includes the following components:

[0110] Multimodal perception module, which is responsible for collecting and perceiving the user's emotional information. Through a variety of sensors and input devices, the system can obtain the user's physiological, psychological and behavioral data in real time. The perception methods include:

[0111] Physiological signal collection: such as heart rate, skin electrical response, breathing rate, etc. These physiological signals can reflect the user's emotional fluctuations.

[0112] Facial expression recognition: Use cameras and facial expression recognition technology to determine the user's emotional state, such as happiness, anger, sadness, etc.

[0113] Speech analysis: Infer the user's emotions by analyzing the user's voice emotions (such as tone, speaking speed, volume, etc.).

[0114] Behavioral analysis: Monitoring user behavior patterns, such as body movements or posture changes, through cameras or other devices can also provide a strong basis for sentiment analysis.

[0115] The emotion analysis and feedback engine is based on artificial intelligence and machine learning algorithms. It combines the data obtained by the perception module to analyze the user's emotional state in real time and determine the cause and type of their emotional changes. The emotion analysis model will classify the user's emotional fluctuation data and give corresponding feedback strategies.

[0116] Emotion Regulation Strategy Module: Based on the emotion analysis results, this module generates personalized regulation strategies. These strategies include:

[0117] Physiological feedback helps users regain calmness by regulating their breathing, heart rate and other physiological indicators. For example, breathing exercises or deep relaxation music can be used to help users relax.

[0118] Cognitive behavioral regulation provides cognitive behavioral therapy-style thinking adjustments to help users look at things more positively and thus change negative emotions.

[0119] Sensory stimulation adjusts the user's emotional state through multimodal stimulation such as vision and hearing.

[0120] In order to enhance the intelligence and scalability of the system, the emotional feedback regulation system uses cloud platform and networking technology and has the following functions:

[0121] Data storage and analysis: All emotional data, feedback strategies, user behavior data, etc. will be uploaded to the cloud platform for big data analysis to explore potential patterns and trends and help further optimize emotional regulation strategies.

[0122] Working in conjunction with other devices, the system can connect with smart home devices, wearable devices, etc., and automatically adjust environmental parameters to suit the user's emotional needs.

[0123] All collected data is processed through machine learning models and emotion recognition algorithms. These algorithms can identify different types of emotions (such as anxiety, anger, happiness, sadness, etc.) and analyze the reasons for emotional changes. For example, an accelerated heart rate indicates that the user is anxious, an increased skin electrical response means that the user is in a state of tension, and voice analysis reveals the user's frustration. Based on the results of the emotion analysis, the system automatically selects the most appropriate adjustment strategy. For example, if the user is anxious, the system will help the user calm down by playing relaxing music, adjusting the ambient temperature, or suggesting deep breathing exercises. If the user is depressed, the system will recommend positive self-talk or stimulate positive emotions through virtual reality scenes.

[0124] When the system detects a user's mood swings, it can connect to other devices or systems to form a linkage response. For example, if the user feels anxious, the system can automatically adjust the color of the smart lights, play soothing music, or predict other support the user needs (such as health advice or rest advice) based on the data provided by the cloud platform. Collaboration between different emotion regulation systems can also bring a more personalized and efficient regulation experience.

[0125] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A multimodal emotion feedback regulation system, characterized in that: Includes the following modules: An emotion recognition module, used to obtain the user's emotional state through multimodal data, the multimodal data including but not limited to physiological signals, voice signals, facial expressions, action recognition and text analysis results; The emotion analysis and modeling module is used to analyze and model the user's emotional state through a deep learning model based on the acquired multimodal data, and generate time series data of the user's emotions; An emotion regulation module is used to provide targeted emotion regulation feedback based on the user's emotional state, and the feedback methods include but are not limited to voice feedback, visual feedback, tactile feedback or environmental adjustment; Feedback optimization module, which is used to optimize the emotion regulation feedback strategy through reinforcement learning algorithm to maximize the positive change of the user's emotional state; The user interface module is used to display the user's real-time emotional state and regulation effect in a visual way, and provide personalized emotion regulation suggestions.

2. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The emotion recognition module includes the following submodules: A physiological signal acquisition unit, used to collect the user's physiological data through sensors, the physiological data including but not limited to heart rate, skin electrical response, electroencephalogram signal, etc.; A speech signal analysis unit, used to perform emotion classification on the user's speech signal through speech recognition and speech emotion analysis models, and extract emotional features from the speech; Facial expression recognition unit, used to identify the emotional changes on the user's face, such as happiness, anger, sadness, etc., through computer vision technology and facial recognition algorithms; The motion recognition unit is used to capture the user's motion and posture through sensors or cameras, and recognize the user's emotional fluctuations based on the motion data.

3. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The sentiment analysis and modeling module adopts a fusion model to jointly analyze multiple modal data, specifically through the following steps: Data preprocessing: standardizing the data of each modality, including but not limited to normalization and missing value filling; The emotion classification model uses deep learning methods such as convolutional neural networks and long short-term memory networks to perform emotion recognition based on processed multimodal data and output emotion categories; Emotion intensity assessment, based on the fusion of modal data, predicts the emotion intensity through a regression model, where the emotion intensity prediction formula is as follows: Emotional intensity = α1HR+α2EDA+α3voice emotion feature+α4facial expression feature+α5action feature Among them, α1, α2, α3, α4, and α5 are the weight coefficients of the model.

4. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The emotion regulation module provides a variety of adjustment feedback methods based on the user's real-time emotional state, including: The voice feedback unit generates voice feedback with emotion regulation effect through speech synthesis technology according to the user's emotional state. The content and tone of the voice feedback are personalized according to the user's emotional characteristics; The visual feedback unit presents different colors, graphics or animations through display screens, LED lights and other devices according to the user's emotional state to adjust the user's emotions; The tactile feedback unit provides tactile feedback to users through vibration, temperature change, etc. to help users adjust their emotions; The environmental adjustment unit improves the user's emotional state by adjusting the environment around him.

5. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The feedback optimization module optimizes the feedback strategy based on the reinforcement learning algorithm, specifically: Reward function, which defines the reward function of the feedback strategy, gives rewards based on changes in the user's emotional state. The formula is as follows: R=ΔEmotional Intensity·w1+ΔEmotional Stability·w2 Among them, Δemotional intensity is the change in the user's emotional intensity, Δemotional stability is the amplitude of emotional fluctuation, and w1 and w2 are weight coefficients.

6. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The user interface module is used to display the user's real-time emotional state and adjustment effect, including: Real-time emotional state visualization unit, which displays the user's current emotional state through a dynamic dashboard, including information such as emotional category and emotional intensity; The regulation effect display unit displays the effect of feedback regulation in real time, such as the change curve of emotion intensity, the change of emotion stability, etc. The emotion regulation suggestion generation unit provides personalized emotion regulation suggestions based on the user's emotional fluctuations and historical data, including recommended regulation methods, feedback types and frequencies, etc.

7. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The system further includes an emotion prediction module for predicting future emotion change trends based on the user's historical emotion data, specifically: The emotional trend prediction model predicts future emotional change trends based on the user's historical emotional data through a time series prediction algorithm. The formula used by the emotional trend prediction model is as follows: E t+1 =f(E t ,X t ,θ) Where: E t is the emotional intensity at time t, X t is the environment variable at time t, θ is the model parameter and f is the emotion prediction function.

8. A multimodal emotion feedback regulation system according to claim 1, characterized in that: The system has an automatic learning function and can continuously adjust the regulation strategy based on user feedback and historical data.

9. The multimodal emotion feedback regulation system according to claim 1, characterized in that: The emotional feedback system can adjust the feedback strategy in real time when receiving external stimulation.

10. The multimodal emotion feedback regulation system according to claim 1, characterized in that: The system can be connected to other devices or systems through a cloud platform to obtain external data or work in collaboration with other emotion regulation systems.

Images