Environment adjusting system with emotion intelligence

Abstract

The invention relates to the technical field of intelligent adjustment, and discloses an environment adjustment system with emotional intelligence, which comprises a data acquisition module used for acquiring language signal data of a user, including language data and voice signal data; the language processing module is used for performing text preprocessing on the language data to obtain language keywords; the emotion score determining module is also used for determining a language emotion score of the user according to the emotion knowledge base and the language keywords based on the optimized language model; the voice processing module is used for performing feature extraction on the voice signal data, determining feature data of the voice signal data, and determining a voice emotion score of the user according to the feature data based on a voice model; and the environment adjusting module is used for determining a comprehensive emotion score of the user according to the language emotion score and the voice emotion score, and adjusting environment light configuration of the user according to the comprehensive emotion score. According to the invention, the emotion of the user can be sensed in real time and indoor light can be automatically adjusted according to the emotion so as to better meet personalized requirements of the user.

Description

Technical Field

[0001] The present invention relates to the field of intelligent regulation technology, and in particular to an environment regulation system with emotional intelligence. Background Art

[0002] With the development of smart home technology, the ambient light adjustment system has become an important device to improve living comfort. Existing ambient light adjustment systems mostly rely on manual operation of users or preset scene modes for adjustment, and lack the ability to perceive and respond to users' emotional states in real time. Therefore, it is necessary to design an ambient light adjustment system with emotional intelligence that can perceive users' emotions in real time and automatically adjust indoor lighting accordingly to better meet users' personalized needs. Summary of the invention

[0003] The purpose of the present invention is to provide an environmental adjustment system with emotional intelligence to solve the problem that existing environmental lighting adjustment systems mostly rely on manual operation of the user or preset scene modes for adjustment and lack the ability to perceive and respond to the user's emotional state in real time.

[0004] The present invention provides an environment adjustment system with emotional intelligence, characterized in that it includes:

[0005] A data acquisition module, used to acquire language signal data of the user, wherein the language signal data includes language data and voice signal data;

[0006] A language processing module is used to perform text preprocessing on the language data, convert the language data into text data, and perform word segmentation and stop word removal on the text data to obtain language keywords; the language processing module is preset with an emotional knowledge base, and is also used to determine the user's language emotion score based on the optimized language model, the emotional knowledge base and the language keywords;

[0007] A speech processing module, used to extract features from the speech signal data, determine feature data of the speech signal data, and determine a speech emotion score of the user according to the feature data based on a speech model;

[0008] The environment adjustment module is used to determine the user's comprehensive emotion score according to the language emotion score and the voice emotion score, and adjust the user's environment lighting configuration according to the comprehensive emotion score.

[0009] Preferably, the emotion knowledge base is constructed based on Plutchik's emotion wheel theory, and the emotion knowledge base includes several emotion types, emotion words, emotion relations and emotion intensities;

[0010] The emotion type corresponds to a plurality of emotion words, the emotion words correspond to a unique emotion intensity, and the emotion relationship is the relationship between the emotion types.

[0011] Preferably, when the language processing module is used to determine the user's language sentiment score based on the optimized language model according to the sentiment knowledge base and language keywords, it includes:

[0012] Searching the language keyword in the emotion knowledge base to obtain the emotion type and emotion intensity of the emotion vocabulary that is the same as the language keyword;

[0013] The optimized language model is used to determine the user's language emotion score according to the emotion type and emotion intensity.

[0014] Preferably, the optimized language model is optimized according to the prompt word project, specifically:

[0015] Obtain the content of test data and sentiment knowledge base;

[0016] Constructing basic prompts according to the content of the sentiment knowledge base, wherein the basic prompts include user input, relevant information and sentiment analysis;

[0017] Adding guiding content to the basic prompt, performing sentiment analysis on the test data according to a language model, and obtaining a sentiment analysis result;

[0018] The sentiment analysis result is compared with the actual result of the test data, and the guiding content is optimized according to the comparison result until the sentiment analysis result of the language model meets the conditions, thereby obtaining an optimized language model.

[0019] Preferably, the speech processing module, when used to extract features from the speech signal data and determine feature data of the speech signal data, includes:

[0020] The characteristic data include pitch value, volume value, speech rate, formant and Mel frequency cepstral coefficient;

[0021] Determine the pitch value of the user in the voice signal data by using an autocorrelation method and a frequency domain analysis method;

[0022] Determine the volume value of the user in the voice signal data by using short-time energy and signal envelope;

[0023] Determining the number of syllables per second in the voice signal data, and determining the user's speaking speed based on the number of syllables;

[0024] Determine the formant of the user in the speech signal data using linear predictive coding;

[0025] Perform Fourier transform and Mel frequency filtering on the speech signal data to determine Mel frequency cepstrum coefficients of the user in the speech signal data.

[0026] Preferably, when the speech processing module is used to determine the speech emotion score of the user based on the speech model and the feature data, it includes:

[0027] The speech model includes an input layer, a convolutional layer, a recurrent layer, an attention mechanism, a fully connected layer and an output layer;

[0028] The input layer generates a feature matrix from the feature data and inputs the matrix into the network;

[0029] The convolution layer performs preliminary processing on the feature matrix to extract the spatiotemporal features of the feature matrix;

[0030] The recurrent layer processes the spatiotemporal features of the feature matrix using a bidirectional long short-term memory network to determine the time dependency of the feature matrix;

[0031] The attention mechanism performs weighted processing on the time dependency of the feature matrix to determine the weight of the feature data;

[0032] The fully connected layer processes the speech signal data according to the weight of the feature data to determine the speech emotion score of the user corresponding to the speech signal data;

[0033] The output layer outputs the speech emotion score of the user.

[0034] Preferably, when the environment adjustment module is used to determine the user's comprehensive emotion score according to the language emotion score and the voice emotion score, it includes:

[0035] Determine a score difference between the language emotion score and the voice emotion score, and compare the language emotion score with the voice emotion score;

[0036] If the score difference is greater than the preset score difference, then when the language emotion score is greater than the voice emotion score, the user's comprehensive emotion score is the language emotion score; when the language emotion score is less than the voice emotion score, the user's comprehensive emotion score is the voice emotion score;

[0037] If the score difference is less than or equal to the preset score difference, a weighted average is performed on the language emotion score and the voice emotion score to determine a comprehensive emotion score of the user.

[0038] Preferably, when the environment adjustment module is used to perform weighted averaging on the language emotion score and the voice emotion score to determine the comprehensive emotion score of the user, it is calculated according to the following formula:

[0039]

[0040] Among them, M z represents the comprehensive sentiment score, a represents the weight coefficient of the language sentiment score, M L represents the language sentiment score, b represents the weight coefficient of the speech sentiment score, M V represents the speech emotion score, and C represents the normalization coefficient.

[0041] Preferably, when the environment adjustment module is used to adjust the user's environment lighting configuration according to the comprehensive emotion score, it includes:

[0042] The ambient lighting configuration includes the light intensity, color temperature and lighting cycle of the light;

[0043] Mathematically modeling the ambient lighting configuration and the comprehensive emotion score and forming a virtual environment;

[0044] Training the agent based on the virtual environment to obtain a deep reinforcement learning model, wherein the deep reinforcement learning model includes the agent, the virtual environment, a state space, an action space, and a reward function;

[0045] An adjustment strategy for the user's ambient lighting configuration is determined based on the deep reinforcement learning model.

[0046] Preferably, the state space is S=[M z , I, T, P], where S represents the state space, M z represents the comprehensive emotional score, I represents the light intensity, T represents the color temperature, and P represents the light period;

[0047] The action space is A=[ΔI, ΔT, ΔP], where A represents the action space, ΔI represents adjusting the light intensity, ΔT represents adjusting the color temperature, and ΔP represents adjusting the light cycle;

[0048] The reward function is expressed as R = M z new -M z old , where M z new represents the comprehensive sentiment score after executing the action, M z-old Represents the comprehensive sentiment score before performing the action.

[0049] Compared with the prior art, the beneficial effect of the present invention is that the present invention judges the user's emotional state according to voice data through deep reinforcement learning, and automatically adjusts environmental parameters through artificial intelligence to provide users with a comfortable and pleasant living and working environment. By acquiring the user's language signal data, the present invention can accurately perceive the user's emotional state, realize personalized environmental adjustment, and enhance user experience; combine the analysis of language and voice signals, comprehensively consider the user's emotional expression, and improve the accuracy and comprehensiveness of emotion recognition; timely adjust the ambient lighting configuration according to the user's emotional score, create an environmental atmosphere that matches the user's emotions, help alleviate the user's emotional pressure, and improve comfort. Using a preset emotional knowledge base and optimized language models and voice models, it is possible to interpret user emotions more accurately and provide a reliable basis for environmental adjustment. It can also be dynamically adjusted according to the emotional responses of different users to adapt to various scenarios and user needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0051] Figure 1 It is a functional block diagram of an environmental adjustment system with emotional intelligence according to the present invention. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0053] like Figure 1 As shown, the present invention provides an environment adjustment system with emotional intelligence, comprising:

[0054] A data acquisition module, used to acquire language signal data of the user, wherein the language signal data includes language data and voice signal data;

[0055] A language processing module is used to perform text preprocessing on the language data, convert the language data into text data, and perform word segmentation and stop word removal on the text data to obtain language keywords; the language processing module is preset with an emotional knowledge base, and is also used to determine the user's language emotion score based on the optimized language model, the emotional knowledge base and the language keywords;

[0056] A speech processing module, used to extract features from the speech signal data, determine feature data of the speech signal data, and determine a speech emotion score of the user according to the feature data based on a speech model;

[0057] The environment adjustment module is used to determine the user's comprehensive emotion score according to the language emotion score and the voice emotion score, and adjust the user's environment lighting configuration according to the comprehensive emotion score.

[0058] In this embodiment, the data acquisition module includes a microphone, which captures voice signals through the microphone, and uses voice emotion recognition technology to analyze the pitch, volume and speech speed changes of the voice to identify the speaker's emotions.

[0059] In some embodiments of the present application, the emotion knowledge base is constructed based on the Plutchik emotion roulette theory, and the emotion knowledge base includes several emotion types, emotion words, emotion relations and emotion intensities;

[0060] The emotion type corresponds to a plurality of emotion words, the emotion words correspond to a unique emotion intensity, and the emotion relationship is the relationship between the emotion types.

[0061] In this embodiment, the emotion knowledge base is constructed based on Plutchik's emotion roulette theory. This knowledge base contains a variety of different emotion types, which correspond to a series of specific emotion words. Each emotion word has a unique emotion intensity value, which reflects the intensity of the emotion expressed by the word. In addition, the emotion knowledge base also covers the relationship between different emotion types, which reveals the mutual connection and influence between them. Through this structure, the emotion knowledge base can describe and analyze the complexity and diversity of human emotions in detail.

[0062] Robert Plutchik proposed the emotion wheel theory, which is a model for describing and understanding human emotions. The emotion wheel divides eight basic emotions into opposing pairs, including joy and sadness, trust and disgust, fear and anger, surprise and expectation. These basic emotions are arranged in a circular chart, showing the relationship and intensity level between them. Robert Plutchik believes that these basic emotions are primitive emotions that are universal in all cultures and can be combined into more complex emotional states. For example, joy and trust can be combined into love, and fear and surprise can be combined into shock. The emotion wheel not only shows the opposition relationship of basic emotions, but also shows the intensity changes of emotions, from low intensity to high intensity, such as from sadness to depression to despair. Plutchik's emotion wheel theory provides a systematic framework for emotion research and is widely used in psychology research, human-computer interaction, affective computing and other fields. Through this theory, people can better understand and analyze the complexity of emotions and their manifestations in different situations.

[0063] It is understandable that the data in the emotional knowledge base comes from existing emotional dictionaries, such as the NRC emotional dictionary or WordNet-Affect, which contain a large number of emotional words and related information and can be used as part of the knowledge base. In addition, corpora containing emotional tags can also be used, such as IEMOCAP, RAVDESS, CREMA-D, etc.

[0064] In some embodiments of the present application, when the language processing module is used to determine the user's language emotion score based on the optimized language model according to the emotion knowledge base and language keywords, it includes: searching the language keywords in the emotion knowledge base to obtain the emotion type and emotion intensity of the emotion vocabulary that is the same as the language keyword; determining the user's language emotion score according to the emotion type and emotion intensity through the optimized language model.

[0065] In this embodiment, the language processing module is designed to evaluate the user's language sentiment score based on an optimized language model. This process involves the use of a sentiment knowledge base and language keywords. Specifically, first, the system will perform a detailed search operation on the language keywords input by the user in the sentiment knowledge base. Through this search step, the system can find sentiment words that match the input language keywords, and further obtain the sentiment type and sentiment intensity corresponding to these sentiment words. The sentiment type usually refers to the category of emotion, while the sentiment intensity reflects the intensity of the emotion, such as strong, medium, mild, etc. Through this information, the system can more accurately understand the emotional tendency and intensity contained in the user's language. Next, the system will use the optimized language model to comprehensively consider these sentiment types and sentiment intensities to determine the user's language sentiment score. The optimized language model can improve the accuracy and reliability of the score. In this way, the system can provide the user with a quantitative language sentiment score, thereby helping the user to better understand the emotional components in his or her language expression.

[0066] In some embodiments of the present application, the optimized language model is optimized according to the prompt word project, specifically: obtaining test data and the content of the emotional knowledge base; constructing basic prompts according to the content of the emotional knowledge base, and the basic prompts include user input, relevant information and emotional analysis; adding guiding content to the basic prompts, and performing emotional analysis on the test data according to the language model to obtain emotional analysis results; comparing the emotional analysis results with the actual results of the test data, and optimizing the guiding content according to the comparison results, until the emotional analysis results of the language model meet the conditions, and obtaining the optimized language model.

[0067] It is understandable that cue word engineering is a key technology to optimize and guide natural language processing models to generate the desired output. In language models, cue word engineering is used to guide the model to retrieve relevant information from the knowledge base and generate sentiment analysis results. The following are detailed steps and specific examples showing how to design cue word engineering to guide language models for sentiment analysis.

[0068] The goals of the prompt word project are: 1. Clarify the task: clearly tell the model that the task to be performed is sentiment analysis. 2. Provide context: provide enough context information to help the model understand user input and retrieved information. 3. Guide generation: guide the generation model to generate reasonable and accurate sentiment analysis results.

[0069] The specific steps include: Step 1: Determine the tasks and goals, make it clear that the task to be performed is sentiment analysis, and define the sentiment labels or categories to be used (such as the eight basic emotions in Robert Plutchik's sentiment roulette theory).

[0070] Step 2: Design basic prompts, which should include user input, retrieved information, and a clear description of the sentiment analysis task. For example:

[0071] i. User input: {user_input}

[0072] ii. Relevant information: {relevant_info}

[0073] iii. Sentiment analysis.

[0074] Step 3: Add guidance content to the basic prompts to help the model understand how to extract emotional information from the context.

[0075] Step 4: Optimize and refine the prompts. Based on the actual results, optimize the prompt content to ensure that the model can generate accurate sentiment analysis results. For example, you can provide more contextual information or clarify the format and content that the model should output.

[0076] Example result 1: Sentiment analysis output:

[0077] i. User input: Last weekend I went on an outing with my family. The weather was great and we had a lot of fun.

[0078] ii.Related information: Joy is a feeling ofgreatpleasure and happiness.Joy andtrust can combine to form love.High intensityjoy is known as ecstasy.

[0079] iii. Sentiment Analysis: Based on the user input and related information, you experienced a strong feeling of joy and happiness. Good weather and enjoyable activities may have led to high positive emotions, which can be described as high intensity ecstasy.

[0080] Example result 2: Sentiment analysis output:

[0081] i. User input: Today was very busy at work, but I completed all my tasks and felt very fulfilled and satisfied.

[0082] ii.Related information:Achievement and satisfaction are feelings offulfillmentand contentment.High intensity satisfaction is known as pride.

[0083] iii. Sentiment analysis: Based on user input and related information, you feel a sense of accomplishment and satisfaction. After completing all tasks, this sense of satisfaction may increase to a high level of pride.

[0084] Example result 3: Sentiment analysis output:

[0085] i. User input: My friend and I had a fight over a trivial matter, and now I feel very regretful and frustrated.

[0086] ii.Related information: Regret and disappointment are feelings associated with sadness.High intensity sadness can lead to despair.

[0087] iii. Sentiment Analysis: Based on the user input and related information, you feel a sense of regret and frustration. These emotions are associated with sadness and can lead to a sense of hopelessness if the intensity increases.

[0088] In some embodiments of the present application, the speech processing module, when used to extract features from the speech signal data and determine the feature data of the speech signal data, includes: the feature data includes pitch value, volume value, speaking rate, resonance peak and Mel-frequency cepstral coefficient; using autocorrelation method and frequency domain analysis method to determine the pitch value of the user in the speech signal data; using short-time energy and signal envelope to determine the volume value of the user in the speech signal data; determining the number of syllables per second in the speech signal data, and determining the user's speaking rate based on the number of syllables; using linear predictive coding to determine the resonance peak of the user in the speech signal data; performing Fourier transform and Mel-frequency filtering on the speech signal data to determine the Mel-frequency cepstral coefficient of the user in the speech signal data.

[0089] In this embodiment, pitch feature extraction is an important task in the fields of sound information retrieval, speech processing, etc. The autocorrelation method is a time domain analysis method suitable for determining the periodicity and frequency of speech signals. The frequency domain analysis method is mainly based on Fourier transform, which observes the periodic characteristics of audio by converting to the frequency domain, thereby obtaining pitch features.

[0090] When processing speech signal data, volume characteristics are important parameters for describing the strength of audio signals, and are crucial for tasks such as speech recognition and audio classification. Short-time energy analysis characterizes the volume of an audio signal by calculating the energy of the audio signal within a specific time frame. Signal envelope extraction focuses more on obtaining the peak value changes of the audio signal, thereby obtaining the dynamic characteristics of the volume. These two methods have their own advantages, and their combined use can more comprehensively describe the volume characteristics of the audio signal.

[0091] Linear Predictive Coding (LPC) is an efficient speech signal processing technology that analyzes the spectral characteristics of speech signals by modeling them as an autoregressive (AR) model. This method uses a linear combination of several past speech sample points to predict the current sample point, thereby accurately estimating speech parameters. In practical applications, LPC can not only be used to extract formant features, but can also be widely used in speech recognition, speech synthesis, noise elimination and other fields.

[0092] Mel-scale Frequency Cepstral Coefficients (MFCC) are important acoustic features widely used in speech recognition, speaker recognition and other fields. MFCC extracts spectral characteristics from speech signals through multiple steps, including preprocessing, fast Fourier transform (FFT), application of Mel filter banks, logarithmic operations, discrete cosine transform (DCT), and dynamic feature extraction. These steps together constitute a complex process aimed at accurately capturing subtle changes and acoustic characteristics in speech signals. The preprocessing of speech signals is the basis of the entire feature extraction process, including pre-emphasis, framing and windowing operations. Pre-emphasis enhances high-frequency components through high-pass filters to make the spectrum flatter, thereby improving the accuracy of analysis. Framing divides the continuous speech signal into short time frames for subsequent processing. The windowing operation is to reduce the signal discontinuity at the frame boundary, usually using a Hamming window or Hanning window to reduce spectral leakage. After preprocessing, the time domain signal is converted to a frequency domain signal through a fast Fourier transform (FFT). FFT is an efficient DFT calculation method. It significantly reduces the amount of calculation by decomposing and recombining signal sequences. The frequency domain signal reveals the frequency composition of the speech signal and provides a basis for the subsequent application of the Mel filter bank. The Mel filter bank is one of the core parts of MFCC feature extraction, which is designed based on the auditory characteristics of the human ear. The Mel frequency scale is nonlinearly related to the actual frequency and is more in line with the human ear's perception of sound. By applying the frequency domain signal obtained by FFT to the Mel filter bank, the response of the human ear to different frequencies can be simulated. The output of each filter represents the signal energy in the frequency band. These energy values ​​are logarithmized and discrete cosine transformed (DCT) are performed to finally obtain the MFCC coefficients.

[0093] In some embodiments of the present application, when the speech processing module is used to determine the user's speech emotion score according to the feature data based on the speech model, it includes: the speech model includes an input layer, a convolutional layer, a recurrent layer, an attention mechanism, a fully connected layer and an output layer; the input layer generates a feature matrix from the feature data and inputs it into the network; the convolutional layer performs preliminary processing on the feature matrix to extract the spatiotemporal features of the feature matrix; the recurrent layer uses a bidirectional long short-term memory network to process the spatiotemporal features of the feature matrix to determine the time dependency of the feature matrix; the attention mechanism performs weighted processing on the time dependency of the feature matrix to determine the weight of the feature data; the fully connected layer processes the speech signal data according to the weight of the feature data to determine the speech emotion score of the user corresponding to the speech signal data; and the output layer outputs the user's speech emotion score.

[0094] In this embodiment, the speech processing module is designed to evaluate the user's speech emotion score by analyzing feature data based on the speech model. The module contains multiple key components, each of which plays a specific role to ensure accurate emotion scoring. First, the speech model consists of multiple hierarchical structures, including input layer, convolution layer, recurrent layer, attention mechanism, fully connected layer and output layer. The role of the input layer is to receive feature data and convert it into a feature matrix for further processing. This feature matrix is ​​then input into the network to lay the foundation for subsequent processing steps. The convolution layer uses a convolutional neural network (CNN) to perform preliminary processing on the input features and extract high-level spatiotemporal features. The convolution layer can capture the local correlation between features. The processing results of the convolution layer provide basic data for the recurrent layer, which further uses a bidirectional long short-term memory network (BiLSTM) to perform in-depth processing on these spatiotemporal features. BiLSTM is good at capturing the time dependency in sequence data, so as to better understand the dynamic characteristics of speech signals. The attention mechanism plays an important role in the processing process. It weights the time dependency of the feature matrix to determine the weights of different feature data. This weighted processing helps the model focus on features that have a greater impact on the sentiment score, highlighting key sentiment information and thus improving the accuracy of the score. After being processed by the attention mechanism, the weights of the feature data are passed to the fully connected layer. The fully connected layer further processes the speech signal data according to the weights of the feature data. The role of this layer is to integrate all weighted feature information to determine the speech sentiment score of the user corresponding to the speech signal data. Finally, the output layer outputs the calculated speech sentiment score of the user to provide intuitive sentiment feedback to the user.

[0095] It is understandable that this application requires a large number of annotated data sets when performing speech processing. The following are some commonly used off-the-shelf speech emotion recognition data sets:

[0096] IEMOCAP (Interactive Emotional Dyadic Motion Capture Database) IEMOCAP is a widely used emotion recognition dataset that contains conversational speech data with multiple emotion labels. The dataset contains about 12 hours of recordings, covering emotions such as anger, disgust, fear, happiness, sadness, surprise, and neutrality.

[0097] RAVDESS (Ryerson Audio-Visual Database of Emotional Speech and Song)RAVDESS contains emotionally rich speech and singing data with high-quality audio and video recordings. The dataset includes speech samples of 24 actors (12 men and 12 women), covering emotions such as calm, happy, sad, angry, afraid, disgusted, and surprised.

[0098] CREMA-D (Crowd-sourced Emotional Multimodal Actors Dataset)CREMA-D contains the speech and facial expression data of actors in different emotional states. The dataset includes 91 actors and covers emotions such as anger, disgust, fear, happiness, neutrality, and sadness.

[0099] EmoDB (Berlin Database of Emotional Speech) EmoDB is a database containing German emotional speech recorded by 10 actors (5 men and 5 women). The dataset covers emotions such as anger, disgust, fear, happiness, sadness, surprise and neutrality.

[0100] MSP-IMPROV (Multimodal Signal Processing-Improvised) MSP-IMPROV is a multimodal emotion dataset that includes speech, video, and sensor data. The dataset includes speech data improvised from scenes, covering a variety of emotional states.

[0101] In some embodiments of the present application, when the environment adjustment module is used to determine the user's comprehensive emotion score according to the language emotion score and the voice emotion score, it includes:

[0102] Determine a score difference between the language emotion score and the voice emotion score, and compare the language emotion score with the voice emotion score;

[0103] If the score difference is greater than the preset score difference, then when the language emotion score is greater than the voice emotion score, the user's comprehensive emotion score is the language emotion score; when the language emotion score is less than the voice emotion score, the user's comprehensive emotion score is the voice emotion score;

[0104] If the score difference is less than or equal to the preset score difference, a weighted average is performed on the language emotion score and the voice emotion score to determine a comprehensive emotion score of the user.

[0105] In this embodiment, the role of the environment adjustment module is to comprehensively evaluate the user's emotional state based on the user's language emotion score and voice emotion score. Specifically, the module first calculates the score difference between the language emotion score and the voice emotion score, and compares the two scores. In this way, the module can determine the relative difference between the language and voice emotion scores. Compare the score difference with the preset score difference. If the calculated score difference is greater than the preset score difference, it means that the difference between the language emotion score and the voice emotion score is large, so it is necessary to determine the user's comprehensive emotion score based on which score is higher. Specifically, if the language emotion score is higher than the voice emotion score, the user's comprehensive emotion score will adopt the language emotion score; conversely, if the voice emotion score is higher than the language emotion score, the user's comprehensive emotion score will adopt the voice emotion score. However, if the calculated score difference is less than or equal to the preset score difference, it means that the difference between the language emotion score and the voice emotion score is small, which is not enough to rely on one of the scores alone to determine the user's comprehensive emotion score. In this case, the module will weighted average the language emotion score and the voice emotion score to obtain a more comprehensive and balanced user emotion score. The weighted average process can set weights according to specific application scenarios and needs to ensure that the final comprehensive emotion score can more accurately reflect the user's true emotional state. In this way, the environment adjustment module can flexibly determine a comprehensive emotion score based on the user's language and voice emotion score, thereby providing a reliable basis for subsequent emotion adjustment and environment optimization.

[0106] In some embodiments of the present application, when the environment adjustment module is used to perform a weighted average of the language emotion score and the voice emotion score to determine the user's comprehensive emotion score, it is calculated according to the following formula:

[0107]

[0108] Among them, M z represents the comprehensive sentiment score, a represents the weight coefficient of the language sentiment score, M L represents the language sentiment score, b represents the weight coefficient of the speech sentiment score, MV represents the speech emotion score, and C represents the normalization coefficient.

[0109] It is understandable that the system needs to output a sentiment score of [-1, 1], where -1 represents an extremely negative sentiment and 1 represents an extremely positive sentiment. In order to convert Robert Plutchik's emotion wheel theory into a sentiment score in the interval [-1, 1], the polarity and intensity of the sentiment need to be considered. The eight basic emotions in the emotion wheel are divided into opposing pairs of emotions, and the sentiment score can be calculated based on the intensity and polarity of these opposing emotions.

[0110] Polarity of Opposing Emotions: Map the opposing emotion pairs in the emotion wheel to the two ends of the emotion score. For example, joy (positive) corresponds to 1 and sadness (negative) corresponds to -1.

[0111] Linear mapping of emotional intensity: linearly map the emotional intensity from low to high to the interval [-1, 1]. For example, low-intensity sadness (such as depression) corresponds to -0.5, and high-intensity sadness (such as despair) corresponds to -1.

[0112] In the present application, the eight basic emotions can be mapped as follows: Joy: positive emotion, from 0 to 1; Sadness: negative emotion, from 0 to -1; Trust: positive emotion, from 0 to 0.7; Disgust: negative emotion, from 0 to -0.7; Fear: negative emotion, from 0 to -0.8; Anger: negative emotion, from 0 to -0.9; Surprise: positive emotion, from 0 to 0.6; Anticipation: positive emotion, from 0 to 0.5; In the present embodiment, the range of the language emotion score and the voice emotion score are both in the interval [-1, 1].

[0113] In some embodiments of the present application, when the environment adjustment module is used to adjust the user's ambient lighting configuration according to the comprehensive emotion score, it includes: the ambient lighting configuration includes the light intensity, color temperature and lighting cycle of the light; mathematically modeling the ambient lighting configuration and the comprehensive emotion score, and forming a virtual environment; training the intelligent agent based on the virtual environment to obtain a deep reinforcement learning model, the deep reinforcement learning model includes the intelligent agent, virtual environment, state space, action space and reward function; determining the adjustment strategy of the user's ambient lighting configuration according to the deep reinforcement learning model.

[0114] In this embodiment, the environment adjustment module is designed to adjust its environment lighting configuration according to the user's comprehensive emotional score. Specifically, the environment lighting configuration includes three key parameters: light intensity, color temperature and light cycle. Specifically, the environment lighting configuration and the comprehensive emotional score are mathematically modeled to construct a virtual environment. This virtual environment can simulate the lighting effects and emotional responses of users in the real world. Next, the agent is trained in this virtual environment. The agent is an algorithmic entity that can learn and make decisions autonomously. Through deep reinforcement learning, the agent can continuously try and optimize its behavior strategy in the virtual environment to achieve the best lighting configuration effect. In this process, a deep reinforcement learning model is constructed, which includes five core components: agent, virtual environment, state space, action space and reward function. The state space defines all possible states that the agent can perceive in the virtual environment, while the action space defines all possible actions that the agent can perform. The reward function is used to evaluate the immediate feedback obtained by the agent after performing a specific action, thereby guiding the agent to optimize in the direction of improving the user's comprehensive emotional score. Finally, based on this deep reinforcement learning model, an effective user environment lighting configuration adjustment strategy can be determined. This strategy can intelligently adjust the light intensity, color temperature and lighting cycle of the light according to the user's real-time emotional score, thereby creating a more comfortable and pleasant environment to meet the user's emotional needs.

[0115] It is understandable that based on the user's current comprehensive emotional score, the system will adjust the configuration of the ambient light. Specifically, the system will adjust the light intensity, color temperature, and light cycle of the light according to the user's current emotional state. For example, when the user is in a low mood, the system may increase the light intensity and adjust the color temperature to provide a warmer environment; when the user is in a high mood, the system may choose a softer light configuration. This adjustment system uses reinforcement learning methods to continuously optimize the light adjustment strategy. Reinforcement learning is a machine learning method that continuously improves the decision-making process through interaction and feedback with the environment. In this system, the system will adjust the lighting configuration strategy based on the user's feedback and emotional changes to achieve the best user experience. Through continuous learning and adjustment, the system can better meet the emotional needs of users.

[0116] In some embodiments of the present application, the state space is S=[M z , I, T, P], where S represents the state space, M z represents the comprehensive emotion score, I represents the light intensity, T represents the color temperature, and P represents the light cycle; the action space is A=[ΔI, ΔT, ΔP], where A represents the action space, ΔI represents adjusting the light intensity, ΔT represents adjusting the color temperature, and ΔP represents adjusting the light cycle; the reward function is expressed as R=Mz-new -M z-old , where M z-new represents the comprehensive sentiment score after executing the action, M z-old Represents the comprehensive sentiment score before performing the action.

[0117] In this embodiment, the task of the agent is to adjust the lighting in the environment (light intensity, color temperature, light cycle) to optimize the mood of the person. The range of the vector in the state space M z ∈[-1, 1]: mood parameter; I∈[0, 100]: light intensity (percentage); T∈[2000, 6500]: color temperature (Kelvin); P∈[0, 24]: photoperiod (hours). In the action space, ΔI: adjust light intensity (increase or decrease); ΔT: adjust color temperature (increase or decrease); ΔP: adjust photoperiod (increase or decrease).

[0118] After executing the action, it also includes: determining the value function Q of executing action A in state S, and updating the Q value using the Q learning algorithm:

[0119] Q(S,A)←Q(S,A)+α[r+γmax A’ Q(S',A')-Q(S,A)];

[0120] Among them, α represents the learning rate, γ represents the discount factor, r represents the immediate reward, and S' represents the new state after executing the action.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that they can still modify or replace the technical solution of the present invention with equivalents, and these modifications or equivalent replacements cannot cause the modified technical solution to deviate from the spirit and scope of the technical solution of the present invention.

[0122] The system provided in the above embodiment is only illustrated by the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be decomposed or combined. For example, the modules in the above embodiment can be combined into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not regarded as improper limitations of the present invention.

[0123] Those skilled in the art should be able to appreciate that the modules and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two, and the programs corresponding to the software modules and method steps can be placed in random access memory (RAM), memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs or any other form of storage medium known in the technical field. In order to clearly illustrate the interchangeability of electronic hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in electronic hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

Claims

1. An environmental adjustment system with emotional intelligence, characterized in that: include: A data acquisition module, used to acquire language signal data of the user, wherein the language signal data includes language data and voice signal data; A language processing module is used to perform text preprocessing on the language data, convert the language data into text data, and perform word segmentation and stop word removal on the text data to obtain language keywords; the language processing module is preset with an emotional knowledge base, and is also used to determine the user's language emotion score based on the optimized language model, the emotional knowledge base and the language keywords; A speech processing module, used to extract features from the speech signal data, determine feature data of the speech signal data, and determine a speech emotion score of the user according to the feature data based on a speech model; The environment adjustment module is used to determine the user's comprehensive emotion score according to the language emotion score and the voice emotion score, and adjust the user's environment lighting configuration according to the comprehensive emotion score.

2. The environment adjustment system with emotional intelligence according to claim 1, characterized in that: The emotion knowledge base is constructed based on Plutchik's emotion wheel theory, and includes several emotion types, emotion words, emotion relations and emotion intensity; The emotion type corresponds to a plurality of emotion words, the emotion words correspond to a unique emotion intensity, and the emotion relationship is the relationship between the emotion types.

3. The environment adjustment system with emotional intelligence according to claim 2, characterized in that: The language processing module, when used to determine the language sentiment score of the user based on the optimized language model according to the sentiment knowledge base and language keywords, includes: Searching the language keyword in the emotion knowledge base to obtain the emotion type and emotion intensity of the emotion vocabulary that is the same as the language keyword; The optimized language model is used to determine the user's language emotion score according to the emotion type and emotion intensity.

4. The environment adjustment system with emotional intelligence according to claim 3, characterized in that: The optimized language model is optimized according to the prompt word project, specifically: Obtain the content of test data and sentiment knowledge base; Constructing basic prompts according to the content of the sentiment knowledge base, wherein the basic prompts include user input, relevant information and sentiment analysis; Adding guiding content to the basic prompt, performing sentiment analysis on the test data according to a language model, and obtaining a sentiment analysis result; The sentiment analysis result is compared with the actual result of the test data, and the guiding content is optimized according to the comparison result until the sentiment analysis result of the language model meets the conditions, thereby obtaining an optimized language model.

5. The environment adjustment system with emotional intelligence according to claim 1, characterized in that: When the speech processing module is used to extract features from the speech signal data and determine feature data of the speech signal data, it includes: The characteristic data include pitch value, volume value, speech rate, formant and Mel frequency cepstral coefficient; Determine the pitch value of the user in the voice signal data by using an autocorrelation method and a frequency domain analysis method; Determine the volume value of the user in the voice signal data by using short-time energy and signal envelope; Determining the number of syllables per second in the voice signal data, and determining the user's speaking speed based on the number of syllables; Determine the formant of the user in the speech signal data using linear predictive coding; Perform Fourier transform and Mel frequency filtering on the speech signal data to determine Mel frequency cepstrum coefficients of the user in the speech signal data.

6. The environment adjustment system with emotional intelligence according to claim 5, characterized in that: The speech processing module, when used to determine the speech emotion score of the user according to the feature data based on the speech model, includes: The speech model includes an input layer, a convolutional layer, a recurrent layer, an attention mechanism, a fully connected layer and an output layer; The input layer generates a feature matrix from the feature data and inputs the matrix into the network; The convolution layer performs preliminary processing on the feature matrix to extract the spatiotemporal features of the feature matrix; The recurrent layer processes the spatiotemporal features of the feature matrix using a bidirectional long short-term memory network to determine the time dependency of the feature matrix; The attention mechanism performs weighted processing on the time dependency of the feature matrix to determine the weight of the feature data; The fully connected layer processes the speech signal data according to the weight of the feature data to determine the speech emotion score of the user corresponding to the speech signal data; The output layer outputs the speech emotion score of the user.

7. The environment adjustment system with emotional intelligence according to claim 2, characterized in that: When the environment adjustment module is used to determine the user's comprehensive emotion score according to the language emotion score and the voice emotion score, it includes: Determine a score difference between the language emotion score and the voice emotion score, and compare the language emotion score with the voice emotion score; If the score difference is greater than the preset score difference, then when the language emotion score is greater than the voice emotion score, the user's comprehensive emotion score is the language emotion score; when the language emotion score is less than the voice emotion score, the user's comprehensive emotion score is the voice emotion score; If the score difference is less than or equal to the preset score difference, a weighted average is performed on the language emotion score and the voice emotion score to determine a comprehensive emotion score of the user.

8. The environment adjustment system with emotional intelligence according to claim 7, characterized in that: When the environment adjustment module is used to perform weighted averaging on the language emotion score and the voice emotion score to determine the user's comprehensive emotion score, the calculation is performed according to the following formula: Among them, M z represents the comprehensive sentiment score, a represents the weight coefficient of the language sentiment score, M L represents the language sentiment score, b represents the weight coefficient of the speech sentiment score, M V represents the speech emotion score, and C represents the normalization coefficient.

9. The environment adjustment system with emotional intelligence according to claim 1, characterized in that: The environment adjustment module, when used to adjust the user's environment lighting configuration according to the comprehensive emotion score, includes: The ambient lighting configuration includes the light intensity, color temperature and light cycle of the light; Mathematically modeling the ambient lighting configuration and the comprehensive emotion score and forming a virtual environment; Training the agent based on the virtual environment to obtain a deep reinforcement learning model, wherein the deep reinforcement learning model includes the agent, the virtual environment, a state space, an action space, and a reward function; An adjustment strategy for the user's ambient lighting configuration is determined based on the deep reinforcement learning model.

10. The environment adjustment system with emotional intelligence according to claim 9, characterized in that: The state space is S = [M z , I, T, P], where S represents the state space, M z represents the comprehensive emotional score, I represents the light intensity, T represents the color temperature, and P represents the light period; The action space is A=[ΔI, ΔT, ΔP], where A represents the action space, ΔI represents adjusting the light intensity, ΔT represents adjusting the color temperature, and ΔP represents adjusting the light cycle; The reward function is expressed as R = M z-new -M z-old , where M z-new represents the comprehensive sentiment score after executing the action, M z-old Represents the comprehensive sentiment score before performing the action.

Images