Smart home AI ornament based on multi-sensor fusion emotional response

Abstract

The invention discloses a smart home AI ornament based on multi-sensor fusion emotion response, and relates to the technical field of artificial intelligence, and the smart home AI ornament comprises a correlation map construction module which is used for collecting environment parameters and emotion characteristics, calculating a cross correlation coefficient, carrying out Granger causality detection, and constructing an environment emotion correlation map; the emotion model construction module is used for converting the environment emotion association map into an adjacent matrix and a node feature matrix, inputting the adjacent matrix and the node feature matrix into an emotion data prediction model and outputting the prediction probability of each emotion category; the environment parameter adjusting module is used for learning an optimal environment adjusting strategy by using a reinforcement learning algorithm; and the interaction behavior decision module is used for judging the interaction intention of the user, judging the influence degree and the influence direction of the environment adjustment strategy on the user emotion according to the time sequence of the user emotion characteristics collected in real time, dynamically generating the interaction behavior decision of the smart home, and realizing the self-adaptive adjustment of the smart home decoration.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a smart home AI ornament based on multi-sensor fusion emotion response. Background Art

[0002] Smart home refers to the intelligent control and management of home appliances through advanced technical means, using technologies such as the Internet, sensors, and artificial intelligence to improve the quality of life and convenience.

[0003] The Chinese patent application with the publication number CN117492378A discloses a smart home regulation method and device based on emotion perception. The method includes: determining the current emotion state of the user and the set of smart devices corresponding to the target area where the user is located, the set of smart devices including at least one smart device; analyzing the sensory adjustment information corresponding to the user according to the current emotion state, the sensory adjustment information including the adjustment information corresponding to at least one target sensory type; determining at least one target device to be regulated and the regulation parameters corresponding to each target device according to the sensory adjustment information and the set of smart devices; for each target device, controlling the target device to execute the operation corresponding to the regulation parameters according to the regulation parameters corresponding to the target device.

[0004] The existing regulation methods of emotion perception home devices usually only consider the user's emotion itself, ignoring the dynamic association and influence mechanism between environmental factors and the user's emotion. When the environmental parameters change, the emotion regulation strategy of the smart home lacks environmental adaptability. Summary of the Invention

[0005] This application aims to solve at least one of the technical problems in the related art to some extent. For this reason, an object of this application is to propose a smart home AI ornament based on multi-sensor fusion emotion response, which realizes the adaptive adjustment of the smart home ornament.

[0006] One aspect of this application provides a smart home AI ornament based on multi-sensor fusion emotion response, including:

[0007] An association graph construction module, configured to collect the time series of environmental parameters and user emotion characteristics, calculate the cross-correlation coefficient between the environmental parameters and the emotion characteristics, and perform Granger causality detection, and construct an environmental emotion association graph based on the test results;

[0008] An emotion model construction module, configured to convert the environmental emotion association graph into an adjacency matrix and a node feature matrix, input them into an emotion data prediction model, output the prediction probabilities of each emotion category, and regularly update the emotion data prediction model;

[0009] The environmental parameter adjustment module is used to define the environmental parameters and the predicted probabilities of each emotion category as states, define the environmental adjustment parameters of the smart home as actions, design a multi-objective reward function, and use a reinforcement learning algorithm to learn the optimal environmental adjustment strategy;

[0010] The interaction behavior decision module is used to judge the user's interaction intention, judge the influence degree and direction of the environmental adjustment strategy on the user's emotion according to the time series of the user's emotion characteristics collected in real time, and dynamically generate the interaction behavior decision of the smart home.

[0011] The specific method for collecting the time series of environmental parameters and user emotion characteristics, calculating the cross-correlation coefficient between the environmental parameters and emotion characteristics, and performing Granger causality detection is as follows:

[0012] Step S110: Integrate environmental sensors on the smart home to collect environmental parameters in real time. The environmental parameters include light, temperature, and humidity, and are recorded as an environmental parameter sequence in chronological order where T1 is the length of the environmental parameter sequence, t1 is a variable representing the moment, and t1 = {1, 2,..., T1}, represents the i-th environmental parameter at time t1;

[0013] Step S120: Collect the user's facial expression data through a camera, collect the user's physiological signal data through a wearable device, extract the user's emotion characteristics from the facial expression data and physiological signal data, and record the emotion characteristics as an emotion characteristic sequence in chronological order where T2 is the length of the emotion characteristic sequence, t2 is a variable representing the moment, and t2 = {1, 2,..., T2}, represents the j-th emotion characteristic at time t2;

[0014] Step S130: Perform time alignment and resampling on the environmental parameter sequence and the emotion characteristic sequence, and convert them into equal-length time series, denoted as and where T' is the length of the aligned sequence, t is a variable representing the moment, and t = {1, 2,..., T'};

[0015] Step S140: Use the sliding window method to obtain the environmental parameter subsequence and the emotion characteristic subsequence according to the environmental parameter sequence and the emotion characteristic sequence. The preset window length is w1, and the sliding step size is s1, and the environmental parameter subsequence within the k1-th time window is obtained and the emotion characteristic subsequence

[0016] Step S150: For the subsequence within each time window, calculate the cross-correlation coefficient r between the environmental parameter i and the emotion characteristic jij (k1);

[0017] The specific method for constructing the environmental emotion association map based on the test results is as follows:

[0018] Step S160: Based on the vector autoregressive model, construct and train a causality model, perform Granger causality test, and determine whether environmental parameter i is the Granger cause of emotion feature j. If the coefficient γ of environmental parameter i jk is not equal to 0, then environmental parameter i is considered the Granger cause of emotion feature j;

[0019] Step S170: Based on the results of the Granger causality test, construct an environmental emotion association map, define environmental parameters and emotion features as nodes, including environmental parameter nodes and emotion feature nodes. Establish relationship edges between nodes where the cross-correlation coefficient is greater than the correlation coefficient threshold, and add directed edges according to the Granger cause. If node A’ is the Granger cause of node B’, then add a directed edge from node A’ to node B’;

[0020] Step S180: Use the absolute value of the cross-correlation coefficient between nodes as the edge weight of the relationship edge;

[0021] The specific method for converting the environmental emotion association map into an adjacency matrix and a node feature matrix, inputting them into the emotion data prediction model, outputting the prediction probabilities of each emotion category, and regularly updating the emotion data prediction model is as follows:

[0022] Step S210: Represent the environmental emotion association map as an adjacency matrix JA and a node feature matrix JX. The element a of the adjacency matrix JA ij represents the edge weight w i between nodes v j and v ij , that is, a ij = w ij . If there is no edge connection between v i and v j , then a ij = 0. The element x of the node feature matrix JX ik represents the k-th eigenvalue of node v i ;

[0023] Step S220: Based on the environmental parameter subsequences and emotion feature subsequences obtained by dividing the environmental parameter sequence and emotion feature sequence, use the emotion category of the user at the end moment of each time window as the emotion category label of this time window. Use the adjacency matrix and node feature matrix of each time window as the input and the emotion category label of the next time window as the output to construct training samples;

[0024] Step S230: Construct a deep graph neural network model. The deep graph neural network model includes an input layer, a graph convolutional layer, and an output layer. Input the adjacency matrix JA and the node feature matrix JX into the input layer of the deep graph neural network model;

[0025] Step S240: The graph convolutional layer consists of L layers of convolutional neural networks. For the (l + 1)-th layer, use the node feature matrix and the adjacency matrix of the l-th layer as inputs to output the node feature matrix of the (l + 1)-th layer. The graph convolutional layer finally outputs the node feature matrix H of L layers (L) , where L is the number of network layers;

[0026] Step S250: The output layer maps the node feature matrix of the last layer to the prediction probabilities of each emotion category

[0027] Step S260: Use the cross-entropy loss function as the loss function Loss of the deep graph neural network model. Use the backpropagation algorithm and an optimizer to train the deep graph neural network model. Use the loss function to calculate the difference between the prediction probabilities and the true emotion categories, and minimize the value of the loss function. When the loss function converges, the training is completed to obtain a trained emotion data prediction model;

[0028] Step S270: Design a sample buffer pool. When a new training sample is collected, add it to the sample buffer pool, and at the same time remove the earliest training sample from the sample buffer pool to keep the size of the sample buffer pool constant;

[0029] Step S280: Periodically randomly sample a batch of training samples from the sample buffer pool and update the emotion data prediction model. Design a forgetting mechanism to assign an exponentially decaying sample weight w to the training samples in the sample buffer pool t ;

[0030] The specific method of defining the environmental parameters and the prediction probabilities of each emotion category as states, defining the environmental adjustment parameters of the smart home as actions, designing a multi-objective reward function, and using the reinforcement learning algorithm to learn the optimal environmental adjustment strategy is as follows:

[0031] Step S310: Define the user's emotion category and the corresponding prediction probabilities as the user's emotion state, and define the environmental parameters and the emotion state as the state s = (s e , s u ), where s e is the environmental parameter and s u is the emotion state;

[0032] Step 320: Define the environmental adjustment parameters of the smart home as the action a of reinforcement learning. The environmental adjustment parameters include, but are not limited to, air conditioner temperature, light brightness, and background music, denoted as a = (a 1 , a 2 , …, a Dn ), where Dn is the number of types of environmental adjustment parameters, and a Dn is the Dn-th type of environmental adjustment parameter;

[0033] Step S330: Calculate the emotional improvement amount ΔE(s u , a), adjustment cost C(a), and user comfort U(s, a) after the user takes action a, and design a multi-objective reward function R(s, a);

[0034] The specific method for calculating the emotional improvement amount ΔE(s u , a), adjustment cost C(a), and user comfort U(s, a) after the user takes action a, and designing a multi-objective reward function R(s, a) is as follows:

[0035] Step S331: According to the predicted probability p c of the c-th emotion category in the current state and the predicted probability p c ’ of the c-th emotion category after taking action a, calculate the emotional improvement amount ΔE(s u , a) after the user takes action a;

[0036] Step S332: Calculate the adjustment cost C(a) of the environmental parameters according to the value of each environmental adjustment parameter a dn ;

[0037] Step S333: Obtain the user comfort U(s, a) of the user for action a in state s according to the user feedback;

[0038] Step S334: Calculate the multi-objective reward function R(s, a) according to the emotional improvement amount ΔE(s u , a), adjustment cost C(a), and user comfort U(s, a);

[0039] Step S340: Introduce the state transition probability P(s’|s, a), and use the reinforcement learning algorithm to learn the optimal environmental adjustment strategy, with the optimization goal of maximizing the expectation of the cumulative multi-objective reward function;

[0040] The specific method for judging the user's interaction intention and dynamically generating the interaction behavior decision of the smart home according to the time series of the user's emotional characteristics collected in real time to judge the influence degree and direction of the environmental adjustment strategy on the user's emotion is as follows:

[0041] Step S410: Real-time collect the interaction action features between the user and the smart home. Let the duration of the g-th interaction action be Δt g , the force be f g , and the frequency be w g , then the interaction action feature x of the g-th interaction action g = {Δt g , f g , w g}, where g = 1, 2,..., G, and G is the total number of interaction actions;

[0042] Step S420: Judge the user's interaction intention h based on the user's interaction action features. Use a random forest as the classification model, with the interaction action features as the input to predict the user's interaction intention h;

[0043] Step S430: Real-time collect the time series of the user's emotion features before and after executing the environment adjustment strategy. The time series of the emotion features includes the pre-adjustment emotion sequence {ej(ta), ta = 1, 2,..., Ta} and the post-adjustment emotion sequence {ej(tb), tb = 1, 2,..., Tb}, and judge the influence degree and influence direction of the environment adjustment strategy on the user's emotion; where ej(ta) and ej(tb) respectively represent the pre-adjustment and post-adjustment emotion features, and Ta and Tb respectively represent the number of emotion features in the pre-adjustment emotion sequence and the post-adjustment emotion sequence;

[0044] Step S440: Design the smart home interaction behavior decision rules based on the influence direction and the user's interaction intention, and generate the interaction behavior U of the smart home according to the smart home interaction behavior decision rules;

[0045] Step S450: Use the influence degree of the environment adjustment strategy on the user's emotion as the execution strength of the smart home interaction behavior, and output the interaction behavior and its execution strength as the smart home interaction behavior decision;

[0046] The acquisition method of the classification model is as follows:

[0047] Step S421: Obtain the user's interaction action features and the corresponding interaction intention annotated by expert knowledge, perform normalization processing on the interaction action features, and convert the interaction intention into a one-hot encoded vector i h ;

[0048] Step S422: Use the user's interaction action features and interaction intention to form training samples, obtain a data set, and randomly and with replacement extract a training sample subset D r , and randomly select an interaction action feature subset F from all interaction action features r, construct a single decision tree by using the training sample subset and the interaction action feature subset;

[0049] Step S423: Integrate m decision trees from the single decision tree, and define the random forest model as: wherein, represents the prediction result of input X, and h r (X) represents the predicted interaction intention category of the r-th decision tree for input X, represents the interaction intention category with the most votes obtained;

[0050] The specific method for judging the influence degree and influence direction of the environmental adjustment strategy on the user's emotion is:

[0051] Step S431: Calculate the feature mean Ea of the pre-adjustment emotion sequence and the feature mean Eb of the post-adjustment emotion sequence respectively;

[0052] Step S432: Calculate the absolute value of the difference between the feature means of the pre-adjustment emotion sequence and the post-adjustment emotion sequence, which is used to represent the influence degree Iab of the environmental adjustment strategy on the user's emotion, and calculate the sign function of the difference, which is used to judge the influence direction Dab of the environmental adjustment strategy on the user's emotion. When Dab = 1, it represents a positive influence. When Dab = -1, it represents a negative influence. When Dab = 0, it represents no influence.

[0053] The smart home AI ornament based on multi-sensor fusion emotion response proposed in this application has the following advantages compared with the prior art:

[0054] By introducing the environmental emotion correlation map, this application establishes the causal relationship between environmental parameters and emotion characteristics, reveals the influence mechanism of the environment on emotions. Based on the correlation map, the emotion data prediction model can incorporate environmental factors and dynamically adapt to environmental changes, improving the accuracy of emotion prediction when environmental parameters change.

[0055] Based on the emotion prediction result and the current environmental parameters, this application finds the optimal environmental adjustment strategy through the reinforcement learning algorithm, uses the regulation of environmental parameters as a means of adjusting the user's emotion, takes into account multiple objectives while improving the user's emotion, and realizes the collaborative optimization of emotion adjustment of the smart home ornament.

[0056] The proposed interaction behavior decision in this application comprehensively utilizes the environmental emotion correlation map, emotion prediction results and environmental adjustment strategies. By judging the influence of environmental adjustment on the user's emotion and combining the interaction intention, it dynamically generates the most appropriate interaction behavior. The interaction decision can actively adapt to the changes of the user's emotion and environment, and provide timely, accurate and personalized emotional interaction services.

[0057] The interaction behavior decision of this application can provide positive and personalized emotional interaction services for users based on the impact of the environmental adjustment strategy on user emotions. The environmental adjustment strategy is optimized based on the current environmental parameters and the results of user emotion prediction. The prediction model used to predict user emotions synthesizes environmental characteristics, the emotional characteristics of the user's historical time, and the influence mechanism between the two, enabling a dynamic closed-loop collaborative feedback optimization mechanism to be formed among emotion prediction, environmental adjustment, and interaction decision-making, providing continuously optimized emotional responses for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 It is a functional module diagram of the smart home AI ornament based on multi-sensor fusion emotion response provided by this application;

[0059] Figure 2 It is a flowchart of the method for constructing the environmental emotion association graph provided by this application;

[0060] Figure 3 It is a flowchart of the method for constructing the multi-objective reward function provided by this application;

[0061] Figure 4 It is a flowchart of the method for the reinforcement learning algorithm to learn the optimal environmental adjustment strategy provided by this application;

[0062] Figure 5 It is a flowchart of the decision-making method for the smart home to execute the interaction behavior decision provided by this application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] To better understand this application, various aspects of this application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of the exemplary embodiments of this application and do not limit the scope of this application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0064] In the accompanying drawings, for ease of illustration, the size, dimensions, and shape of the elements have been slightly adjusted. The drawings are only examples and are not drawn to scale strictly. As used herein, terms such as "substantially", "approximately", and similar terms are used as terms indicating approximation and not as terms indicating degree, and are intended to illustrate the inherent deviations in measured or calculated values that would be recognized by those of ordinary skill in the art. Additionally, in this application, the order of description of the various step processes does not necessarily represent the order in which these processes occur in actual operation, unless otherwise clearly defined or derivable from the context.

[0065] It should also be understood that expressions such as "including", "comprising", "having", "containing" and / or "consisting of" are open-ended rather than closed-ended expressions in this specification, which means that the stated features, elements and / or components exist, but do not exclude the existence of one or more other features, elements, components and / or their combinations. In addition, when an expression such as "at least one of..." appears after a list of listed features, it modifies the entire list of features, rather than just an individual element in the list. In addition, when describing the embodiments of the present application, the use of "may" means "one or more embodiments of the present application". And the term "exemplary" is intended to refer to an example or illustration.

[0066] Unless otherwise defined, all terms used herein (including engineering terms and technical terms) have the same meaning as the ordinary understanding of those of ordinary skill in the art to which this application belongs. It should also be understood that unless clearly stated in this application, words defined in common dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art, and should not be interpreted in an idealized or overly formal sense.

[0067] It should be noted that, without conflict, the embodiments in this application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and in combination with the embodiments.

[0068] Embodiment 1

[0069] As Figure 1 shown, the smart home AI ornament based on multi-sensor fusion emotion response provided by the present application includes:

[0070] An association graph construction module, configured to collect time series of environmental parameters and user emotion features, calculate the cross-correlation coefficient between the environmental parameters and the emotion features, and perform Granger causality detection, and construct an environmental emotion association graph based on the test results;

[0071] An emotion model construction module, configured to convert the environmental emotion association graph into an adjacency matrix and a node feature matrix, input them into an emotion data prediction model, output the prediction probabilities of each emotion category, and regularly update the emotion data prediction model;

[0072] An environmental parameter adjustment module, configured to define the environmental parameters and the prediction probabilities of each emotion category as states, define the environmental adjustment parameters of the smart home as actions, design a multi-objective reward function, and use a reinforcement learning algorithm to learn the optimal environmental adjustment strategy;

[0073] An interaction behavior decision-making module, which is used to judge the user's interaction intention, judge the degree and direction of the impact of the environmental regulation strategy on the user's emotion according to the time series of the user's emotion characteristics collected in real time, and dynamically generate the interaction behavior decision of the smart home.

[0074] The specific method for collecting the time series of environmental parameters and user emotion characteristics, calculating the cross-correlation coefficient between the environmental parameters and the emotion characteristics, and performing Granger causality detection is as follows:

[0075] Step S110: Integrate environmental sensors on the smart home to collect environmental parameters in real time. The environmental parameters include light, temperature, and humidity, and are recorded as an environmental parameter sequence in chronological order where T1 is the length of the environmental parameter sequence, t1 is a variable representing the moment, and t1 = {1, 2,..., T1}, represents the i-th environmental parameter at time t1;

[0076] Step S120: Collect the user's facial expression data through a camera, collect the user's physiological signal data through a wearable device, extract the user's emotion characteristics from the facial expression data and the physiological signal data, and record the emotion characteristics as an emotion characteristic sequence in chronological order where T2 is the length of the emotion characteristic sequence, t2 is a variable representing the moment, and t2 = {1, 2,..., T2}, represents the j-th emotion characteristic at time t2;

[0077] The emotion characteristics refer to the emotion category to which the user belongs and the degree of this emotion category;

[0078] Furthermore, the collected environmental parameters, user facial expression data, and physiological signal data are transmitted to the smart home system through a wireless communication network, and the user's emotion characteristics are extracted from the user's facial expression data and physiological signal data.

[0079] Step S130: Perform time alignment and resampling on the environmental parameter sequence and the emotion characteristic sequence, and convert them into equal-length time series, which are respectively denoted as and where T' is the length of the aligned sequence, t is a variable representing the moment, and t = {1, 2,..., T'};

[0080] Step S140: Use the sliding window method to obtain the environmental parameter subsequence and the emotion characteristic subsequence according to the environmental parameter sequence and the emotion characteristic sequence. The preset window length is w1, and the sliding step size is s1, and the environmental parameter subsequence within the k1-th time window is obtained and the emotion characteristic subsequence

[0081] Step S150: Calculate the cross-correlation coefficient r between the environmental parameter i and the emotional feature j for each subsequence within each time window ij (k1);

[0082] The formula for calculating the cross-correlation coefficient between the environmental parameter i and the emotional feature j is as follows: where, and are the means of the environmental parameter i and the emotional feature j within the k1-th time window respectively;

[0083] The specific method for constructing the environmental-emotion association map based on the test results is as follows:

[0084] Step S160: Based on the vector autoregressive model, construct and train a causality model, conduct a Granger causality test to determine whether the environmental parameter i is the Granger cause of the emotional feature j. If the coefficient γ of the environmental parameter i jk is not equal to 0, then it is considered that the environmental parameter i is the Granger cause of the emotional feature j;

[0085] The expression of the causality model is: where p is the lag order, α j is the constant term, β jk and γ jk are the coefficient of the emotional feature j and the coefficient of the environmental parameter i respectively, is the emotional feature j at time t - k, is the environmental parameter i at time t - k, is the random disturbance term;

[0086] The causality model is used to describe the dynamic relationship between variables. It indicates that the emotional feature is related not only to its own past values but also to the past values of the environmental parameter;

[0087] The principle of the Granger causality test is: If variable 1 helps to understand the future changes of variable 2, that is, adding the lag term of variable 1 to the model containing the past information of variable 2 itself can improve the prediction ability of variable 2, then variable 1 is called the Granger cause of variable 2;

[0088] The specific method for constructing and training the causality model is as follows:

[0089] Step S161: Set the lag order p of the causality model with the emotional feature as the endogenous variable and the environmental parameter as the exogenous variable, and construct the causality model based on the vector autoregressive model;

[0090] The lag order p is set by those skilled in the art according to experience;

[0091] Step S162: For each endogenous variable, an equation is formed from the expression of the causal relationship model to obtain a system of equations, and the system of equations is transformed into a matrix form to obtain the standard form of the causal relationship model;

[0092] The standard form of the causal relationship model is: Y t = A + B1×Y t-1 +... + B p ×Y t-p + C1×X t-1 +... + C p ×X t-p + E t , where Y t is the value of m1 endogenous variables at time t, m1 represents the number of endogenous variables, A is the constant term of each endogenous variable, B1,..., B p are the elements of the m1×m1 coefficient matrix, B p represents the influence coefficient of the endogenous variable at lag order p on the endogenous variable, Y t-p represents the value of m1 endogenous variables at time t - p, C1,..., C p are the elements of the m1×k1 coefficient matrix, C p represents the influence of the exogenous variable at lag order p on the endogenous variable, k1 is the number of exogenous variables, X t-p represents the value of k1 exogenous variables at time t - p, E t represents the vector of random disturbance terms;

[0093] Step S163: Using the environmental parameter sequence and the emotion feature sequence as training data, the causal relationship model is trained with the training data, and the coefficients of the causal relationship model are estimated by the method of maximum likelihood estimation;

[0094] The coefficients of the causal relationship model include the constant term of the endogenous variable, the m1×m1 coefficient matrix, and the m1×k1 coefficient matrix;

[0095] Step S164: Conduct a significance test on the coefficients of the causal relationship model, determine which coefficients are not zero, and incorporate the corresponding terms of the non-zero coefficients as lag terms into the causal relationship model to obtain the estimated causal relationship model;

[0096] Step S170: Based on the results of the Granger causality test, construct an environmental-emotion association graph, define the environmental parameters and emotion features as nodes, where the nodes include environmental parameter nodes and emotion feature nodes, establish relationship edges between nodes with a cross-correlation coefficient greater than the correlation coefficient threshold, and add directed edges according to the Granger cause. If node A' is the Granger cause of node B', then add a directed edge from node A' to node B';

[0097] The attributes of the environmental parameter nodes include the statistics of each environmental parameter, such as the mean and variance;

[0098] The attributes of the emotion feature nodes include the frequency or intensity of various emotion categories;

[0099] The correlation coefficient threshold is set by those skilled in the art according to experience;

[0100] Step S180: Use the absolute value of the cross-correlation coefficient between nodes as the edge weight of the relationship edge;

[0101] Figure 2 It is a flowchart of the method for constructing the environmental emotion association graph provided by this application;

[0102] The above steps introduce the environmental emotion association graph, establish the causal relationship and correlation between environmental parameters and user emotions, reveal the dynamic influence mechanism of environmental factors on user emotions, and provide a key decision-making basis for emotion prediction and intelligent regulation.

[0103] The specific method of converting the environmental emotion association graph into an adjacency matrix and a node feature matrix, inputting them into the emotion data prediction model, outputting the prediction probabilities of each emotion category, and regularly updating the emotion data prediction model is as follows:

[0104] Step S210: Represent the environmental emotion association graph as an adjacency matrix JA and a node feature matrix JX. The element a of the adjacency matrix JA ij represents the edge weight w i between nodes v j and v ij , that is, a ij = w ij . If there is no edge connection between v i and v j , then a ij = 0. The element x of the node feature matrix JX ik represents the kth eigenvalue of node v i ;

[0105] The rows of the node feature matrix represent the features of a node, and the columns represent the node feature dimensions;

[0106] Step S220: Based on the environmental parameter subsequences and emotion feature subsequences obtained by dividing the environmental parameter sequence and emotion feature sequence, use the emotion category of the user at the end moment of each time window as the emotion category label of this time window. Use the adjacency matrix and node feature matrix of each time window as the input and the emotion category label of the next time window as the output to construct training samples;

[0107] The adjacency matrix and node feature matrix of each time window are updated based on the environmental parameters and emotional characteristics of that time window, and then used as inputs;

[0108] Step S230: Construct a deep graph neural network model, which includes an input layer, a graph convolutional layer, and an output layer. Input the adjacency matrix JA and the node feature matrix JX into the input layer of the deep graph neural network model;

[0109] Step S240: The graph convolutional layer is composed of L layers of convolutional neural networks. For the (l + 1)-th layer, use the node feature matrix and adjacency matrix of the l-th layer as inputs, and output the node feature matrix of the (l + 1)-th layer. The graph convolutional layer finally outputs the node feature matrix H of the L-th layer (L) , where L is the number of network layers;

[0110] The node feature matrix of the (l + 1)-th layer is expressed as: where H (l+1) and H (l) are the node feature matrices of the (l + 1)-th layer and the l-th layer respectively, σ(·) is the activation function, W (l) is the weight matrix of the l-th layer, is the adjacency matrix with self-connections added, is 's degree matrix;

[0111] Step S250: The output layer maps the node feature matrix of the last layer to the predicted probabilities of each emotion category

[0112] The predicted probabilities of the emotion categories are: where f(·) is the softmax activation function, W o is the weight matrix of the output layer, and b o is the bias term;

[0113] The predicted probabilities of the emotion categories are the probability values that the user's emotion belongs to each emotion category;

[0114] Step S260: Use the cross-entropy loss function as the loss function Loss of the deep graph neural network model. Train the deep graph neural network model using the backpropagation algorithm and an optimizer. Use the loss function to calculate the difference between the predicted probabilities and the true emotion categories, and minimize the value of the loss function. When the loss function converges, the training is completed, and a trained emotion data prediction model is obtained;

[0115] Step S270: Design a sample buffer pool. When a new training sample is collected, add it to the sample buffer pool, and at the same time, remove the earliest training sample from the sample buffer pool to keep the size of the sample buffer pool constant;

[0116] Step S280: Regularly randomly sample a batch of training samples from the sample buffer pool and update the emotion data prediction model. Design a forgetting mechanism to assign sample weights \(w\) that decay exponentially with time to the training samples in the sample buffer pool t ;

[0117] The sample weight \(w\) that decays exponentially with time t has the following calculation formula: where \(w\) t is the sample weight at time \(t\), \(t_0\) is the time when the training sample enters the sample buffer pool, and \(\lambda\) is the decay factor;

[0118] The specific value of the decay factor is set by those skilled in the art according to experience, and \(\lambda\in(0,1)\);

[0119] The batch size of randomly sampling a batch of training samples is set by those skilled in the art according to experience;

[0120] Further, in steps S270 and S280, upload the data related to the training samples in the sample buffer pool to the cloud server through a wireless communication network, update the emotion data prediction model on the cloud server, and send the updated model parameters to the local device.

[0121] The above steps convert the environmental emotion association map into an adjacency matrix and a node feature matrix, input them into a deep graph neural network model for emotion prediction, and design a forgetting mechanism for regular model update, realizing end-to-end emotion prediction based on graph-structured data, and significantly improving the accuracy and timeliness of prediction.

[0122] The specific method of defining the environmental parameters and the prediction probabilities of each emotion category as states, defining the environmental adjustment parameters of the smart home as actions, designing a multi-objective reward function, and using a reinforcement learning algorithm to learn the optimal environmental adjustment strategy is as follows:

[0123] Step S310: Define the user's emotion category and the corresponding prediction probability as the user's emotion state, and define the environmental parameters and the emotion state as the state \(s=(s e ,s u ), where \(s e is the environmental parameter and \(s u is the emotion state;

[0124] Step 320: Define the environmental adjustment parameters of the smart home as the actions a of reinforcement learning. The environmental adjustment parameters include, but are not limited to, air conditioner temperature, light brightness, and background music, denoted as a = (a 1 , a 2 , …, a Dn ), where Dn is the number of types of environmental adjustment parameters, and a Dn is the Dn-th type of environmental adjustment parameter;

[0125] The execution of the actions of the reinforcement learning controls the smart home ornaments through a wireless communication network;

[0126] Step S330: Calculate the emotional improvement amount ΔE(s u , a), adjustment cost C(a), and user comfort U(s, a) after the user takes action a, and design a multi-objective reward function;

[0127] The specific method for calculating the emotional improvement amount ΔE(s u , a), adjustment cost C(a), and user comfort U(s, a) after the user takes action a and designing a multi-objective reward function is as follows:

[0128] Step S331: Calculate the emotional improvement amount ΔE(s c , a) after the user takes action a according to the predicted probability p c of the c-th emotional category in the current state and the predicted probability p u ’ of the c-th emotional category after taking action a;

[0129] The calculation formula for the emotional improvement amount is: where w c is the weight coefficient of the c-th emotional category, and C is the total number of emotional categories;

[0130] The weight coefficient of the emotional category is set by those skilled in the art according to experience;

[0131] Step S332: Calculate the adjustment cost C(a) of the environmental parameters according to the value of each environmental adjustment parameter a dn ;

[0132] The calculation formula for the adjustment cost is: where α dn is the cost coefficient of the dn-th environmental adjustment parameter, and a dn is the dn-th environmental adjustment parameter;

[0133] The cost coefficient of the environmental adjustment parameter is set by those skilled in the art according to experience;

[0134] Step S333: Obtain the user comfort U(s,a) of the user for action a in state s based on user feedback;

[0135] The value range of the user comfort U(s,a) is [0,1];

[0136] Step S334: Calculate the multi-objective reward function according to the emotion improvement amount ΔE(s u ,a), adjustment cost C(a), and user comfort U(s,a);

[0137] The calculation formula of the multi-objective reward function is: where β1, β2, β3, and β4 are the weight coefficients for balancing the emotion improvement amount, adjustment cost, user comfort, and the interaction term between the emotion improvement amount and user comfort respectively, λ1 and λ2 are the parameters for controlling the steepness of the Sigmoid function, μ1 and μ2 are the midpoint parameters of the Sigmoid function, and γ is the exponential parameter of the adjustment cost;

[0138] The exponential parameter of the adjustment cost is used to control the non-linearity degree of the cost. The weight coefficients for the emotion improvement amount, adjustment cost, user comfort, the interaction term between the emotion improvement amount and user comfort, the exponential parameter of the adjustment cost, and the midpoint parameters of the Sigmoid function are set by those skilled in the art according to experience;

[0139] Figure 3 is the flowchart of the construction method of the multi-objective reward function provided by this application;

[0140] The above multi-objective reward function performs Sigmoid transformation on the emotion improvement amount and user comfort, which can reflect their characteristics of diminishing marginal utility, that is, when the improvement amount or comfort is already very high, the marginal effect of further improvement will decrease;

[0141] The adjustment cost is in exponential form, which can reflect the non-linear rapid growth of the cost;

[0142] Adding the interaction term between the emotion improvement amount and comfort indicates a positive correlation between the two, that is, the more the emotion is improved, the more comfortable the user may be.

[0143] Step S340: Introduce the state transition probability P(s’|s,a), and use the reinforcement learning algorithm to learn the optimal environment adjustment strategy, with the optimization goal of maximizing the expectation of the cumulative multi-objective reward function;

[0144] The expectation of the cumulative multi-objective reward function is: where Q(s,a) is the expectation of the cumulative multi-objective reward function for taking action a in state s, γ Qis the discount factor, s' is the next state, a' is the next action, is the maximum Q-value of the next state, and Q(s', a') is the expectation of the cumulative multi-objective reward function for taking action a' in state s'. represents the expected value of transferring to the next state s' after taking action a in state s;

[0145] The state transition probability represents the probability of transferring from state s to state s’ after taking action a in state s;

[0146] Figure 4 is the method flowchart for the reinforcement learning algorithm provided by this application to learn the optimal environment regulation strategy;

[0147] The above steps use the environmental parameters and emotion prediction results as states, the environmental regulation parameters as actions, design a multi-objective reward function, and use the reinforcement learning algorithm to learn the optimal environmental regulation strategy, realizing the adaptive optimization of environmental control, and taking into account multiple objectives such as energy consumption and comfort while improving the user's mood.

[0148] The specific method for judging the user's interaction intention and dynamically generating the interaction behavior decision of the smart home according to the time series of the user's emotion characteristics collected in real time is as follows:

[0149] Step S410: Collect the interaction action characteristics between the user and the smart home in real time. Let the duration of the g-th interaction action be Δt g , the strength be f g , and the frequency be w g , then the interaction action characteristic x of the g-th interaction action g ={Δt g , f g , w g}, where g = 1, 2,..., G, and G is the total number of interaction actions;

[0150] Step S420: Judge the user's interaction intention h based on the user's interaction action characteristics. Use a random forest as the classification model, with the interaction action characteristics as the input to predict the user's interaction intention h;

[0151] The acquisition method of the classification model is as follows:

[0152] Step S421: Obtain the user's interaction action characteristics and the corresponding interaction intention annotated by expert knowledge, perform normalization processing on the interaction action characteristics, and convert the interaction intention into a one-hot encoded vector i h ;

[0153] The one-hot encoded vector i h is expressed as: ih = [I(i h = 1), I(i h = 2),..., I(i h = M)], where i h is the one-hot encoded vector of the interaction intention h, M is the number of interaction intention categories, and i h = M indicates that the user's interaction intention category is the M-th interaction intention. I(·) is the indicator function, which takes the value 1 when the condition is true and 0 otherwise;

[0154] Step S422: Construct training samples from the user's interaction action features and interaction intentions to obtain a data set. Randomly draw a training sample subset D r with replacement from the data set, and randomly select an interaction action feature subset F r from all interaction action features, and construct a single decision tree using the training sample subset and the interaction action feature subset;

[0155] Step S423: Integrate mr decision trees from a single decision tree, and define the random forest model as: where, represents the prediction result of the input X, and h r (X) represents the predicted interaction intention category of the r-th decision tree for the input X, represents the interaction intention category with the most votes obtained;

[0156] Step S430: Real-time collect the time series of the user's emotional features before and after executing the environment adjustment strategy. The time series of the emotional features includes the pre-adjustment emotion sequence {ej(ta), ta = 1, 2,..., Ta} and the post-adjustment emotion sequence {ej(tb), tb = 1, 2,..., Tb}, and judge the influence degree and influence direction of the environment adjustment strategy on the user's emotion; where, ej(ta) and ej(tb) respectively represent the pre-adjustment emotional feature and the post-adjustment emotional feature, and Ta and Tb respectively represent the number of emotional features in the pre-adjustment emotion sequence and the post-adjustment emotion sequence;

[0157] The specific method for judging the influence degree and influence direction of the environment adjustment strategy on the user's emotion is:

[0158] Step S431: Calculate the feature mean Ea of the pre-adjustment emotion sequence and the feature mean Eb of the post-adjustment emotion sequence respectively;

[0159] The calculation formula for the feature mean Ea of the pre-adjustment emotion sequence is:

[0160] The calculation formula for the feature mean Eb of the post-adjustment emotion sequence is:

[0161] Step S432: Calculate the absolute value of the difference between the characteristic means of the pre-regulation emotion sequence and the post-regulation emotion sequence, which is used to represent the influence degree Iab of the environmental regulation strategy on the user's emotion. Calculate the sign function of the difference to determine the influence direction Dab of the environmental regulation strategy on the user's emotion. When Dab = 1, it indicates a positive influence; when Dab = -1, it indicates a negative influence; when Dab = 0, it indicates no influence;

[0162] The formula for calculating the influence degree of the environmental regulation strategy on the user's emotion is: Iab = |Ea - Eb|;

[0163] The formula for calculating the influence direction of the environmental regulation strategy on the user's emotion is: Dab = sign(Ea - Eb);

[0164] The value range of the influence direction Dab is {1, 0, -1}. When Ea - Eb > 0, sign(Ea - Eb) = 1; when Ea - Eb < 0, sign(Ea - Eb) = -1; when Ea - Eb = 0, sign(Ea - Eb) = 0;

[0165] Step S440: Design the smart home interaction behavior decision rules based on the influence direction and the user's interaction intention, and generate the interaction behavior U of the smart home according to the smart home interaction behavior decision rules;

[0166] The smart home interaction behavior decision rules are as follows:

[0167] When Dab = 1 and h = soothe, then U = rub hands;

[0168] When Dab = -1 and h = soothe, then U = lie down;

[0169] When Dab = 1 and h = feed, then U = rub legs;

[0170] When Dab = -1 and h = feed, then U = sit down;

[0171] When Dab = 1 and h = play, then U = bark cheerfully;

[0172] When Dab = -1 and h = play, then U = wag tail;

[0173] The smart home is a simulated comfort dog, and the simulated comfort dog is integrated with a variety of sensors for obtaining environmental parameters and user facial expression data.

[0174] Step S450: Use the degree of influence of the environmental adjustment strategy on the user's emotion as the execution intensity of the interaction behavior of the smart home, and output the interaction behavior and its execution intensity as the interaction behavior decision of the smart home;

[0175] The interaction behavior decision of the smart home collects the user's interaction intention and emotion characteristics through a wireless communication network. The wireless communication network is connected to the smart home, and the smart home executes the corresponding interaction behavior decision according to the interaction intention and emotion characteristics;

[0176] As Figure 5 shown, it is the flowchart of the decision method for the smart home to execute the interaction behavior decision provided by this application;

[0177] The above steps dynamically generate personalized interaction behavior decisions by real-time perceiving the user's interaction behavior, judging the interaction intention, and combining the time series analysis of emotion characteristics to analyze the influence of environmental adjustment on emotions. The smart home integrates the perception, adjustment and mutual feedback of the environment and emotions, forming a closed loop with the previous steps, and can provide continuous and personalized emotional assistance services.

[0178] In addition, the parts of the above technical solutions provided in the embodiments of this application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0179] As described in the above specific embodiments, the purpose, technical solutions and beneficial effects of the present invention are further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection scope of the present invention.

Claims

1. A smart home AI ornament based on multi-sensor fusion emotion response, characterized in that, Including: An associated graph construction module, which is used to collect time series of environmental parameters and user emotion features, calculate the cross-correlation coefficient between the environmental parameters and the emotion features, and perform Granger causality detection, and construct an environmental emotion association graph based on the test results; An emotion model construction module, which is used to convert the environmental emotion association graph into an adjacency matrix and a node feature matrix, input them into an emotion data prediction model, output the prediction probabilities of each emotion category, and regularly update the emotion data prediction model; An environmental parameter adjustment module, which is used to define the environmental parameters and the prediction probabilities of each emotion category as states, define the environmental adjustment parameters of the smart home as actions, design a multi-objective reward function, and use a reinforcement learning algorithm to learn the optimal environmental adjustment strategy; An interaction behavior decision-making module, which is used to judge the user's interaction intention, judge the influence degree and direction of the environmental adjustment strategy on the user's emotion according to the time series of the user's emotion features collected in real time, and dynamically generate the interaction behavior decision of the smart home.

2. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 1, wherein The specific method for collecting the time series of environmental parameters and user emotion features, calculating the cross-correlation coefficient between the environmental parameters and the emotion features, and performing Granger causality detection is as follows: Integrate environmental sensors in the smart home to collect environmental parameters in real time. The environmental parameters include light, temperature, and humidity, and are recorded as an environmental parameter sequence in chronological order. Among them, T1 is the length of the environmental parameter sequence, t1 is a variable representing the moment, and t1 = {1, 2,..., T1}. represents the i-th environmental parameter at moment t1. Collect the facial expression data of the user through a camera, collect the physiological signal data of the user through a wearable device, extract the emotional features of the user from the facial expression data and the physiological signal data, and record the emotional features as an emotional feature sequence in chronological order where T2 is the length of the emotional feature sequence, t2 is a variable representing the time, and t2 = {1, 2,..., T2} represents the j-th emotional feature at time t2 Perform time alignment and resampling on the environmental parameter sequence and the emotion feature sequence, and convert them into time series of equal length, denoted as and where T' is the length of the aligned sequence, t is a variable representing the time instant, and t = {1, 2,..., T'}; The sliding window method is used to obtain the environmental parameter subsequence and the emotion feature subsequence according to the environmental parameter sequence and the emotion feature sequence. The preset window length is w1, and the sliding step length is s1, so as to obtain the environmental parameter subsequence within the k1-th time window and the emotion feature subsequence For each subsequence within each time window, calculate the cross-correlation coefficient r between environmental parameter i and emotional feature j ij (k1).

3. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 2, wherein The specific method for constructing the environmental emotion association graph based on the test results is as follows: Based on the vector autoregressive model, construct and train a causal relationship model, conduct a Granger causality test to determine whether the environmental parameter i is the Granger cause of the emotional feature j. If the coefficient γ of the environmental parameter i jk is not equal to 0, it is considered that the environmental parameter i is the Granger cause of the emotional feature j; Based on the results of Granger causality test, construct an environmental emotion association graph, define the environmental parameters and emotion features as nodes, the nodes include environmental parameter nodes and emotion feature nodes, establish relationship edges between the nodes with the cross-correlation coefficient greater than the correlation coefficient threshold, and add directed edges according to the Granger cause. If node A’ is the Granger cause of node B’, then add a directed edge from node A’ to node B’; Take the absolute value of the cross-correlation coefficient between the nodes as the edge weight of the relationship edge.

4. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 3, wherein, The specific method for converting the environmental emotion association graph into an adjacency matrix and a node feature matrix, inputting them into the emotion data prediction model, outputting the prediction probabilities of each emotion category, and regularly updating the emotion data prediction model is as follows: Represent the environmental emotion association graph as an adjacency matrix JA and a node feature matrix JX. The element a of the adjacency matrix JA ij represents the edge weight w i between nodes v j and v ij , that is, a ij = w ij . If there is no edge connecting v i and v j , then a ij = 0. The element x of the node feature matrix JX ik represents the k-th eigenvalue of node v i ; Based on the environmental parameter subsequences and emotion feature subsequences obtained by dividing the environmental parameter sequence and the emotion feature sequence, take the emotion category of the user at the end moment of each time window as the emotion category label of this time window, use the adjacency matrix and node feature matrix of each time window as the input, and the emotion category label of the next time window as the output to construct training samples; Construct a deep graph neural network model, the deep graph neural network model includes an input layer, a graph convolutional layer, and an output layer, and input the adjacency matrix JA and the node feature matrix JX into the input layer of the deep graph neural network model; The graph convolutional layer consists of L layers of convolutional neural networks. For the (l + 1)-th layer, taking the node feature matrix and the adjacency matrix of the l-th layer as inputs, it outputs the node feature matrix of the (l + 1)-th layer. The graph convolutional layer finally outputs the node feature matrix H of the L-th layer (L) , where L is the number of network layers; The output layer maps the node feature matrix of the last layer to the predicted probabilities of each emotion category Take the cross-entropy loss function as the loss function Loss of the deep graph neural network model, use the backpropagation algorithm and an optimizer to train the deep graph neural network model, use the loss function to calculate the difference between the predicted probability and the true emotion category, minimize the value of the loss function, and when the loss function converges, the training is completed to obtain a trained emotion data prediction model; Design a sample buffer pool. When new training samples are collected, add them to the sample buffer pool, and at the same time remove the earliest training samples from the sample buffer pool to keep the size of the sample buffer pool constant; Randomly sample a batch of training samples from the sample buffer pool regularly, update the emotion data prediction model, design a forgetting mechanism, and assign sample weights \(w\) that decay exponentially over time to the training samples in the sample buffer pool t 。 5. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 4, characterized in that, Define the environmental parameters and the predicted probabilities of each emotion category as states, and define the environmental adjustment parameters of the smart home as actions. The specific method for designing a multi-objective reward function and using a reinforcement learning algorithm to learn the optimal environmental adjustment strategy is as follows: Define the user's emotional category and the corresponding prediction probability as the user's emotional state, and define the environmental parameters and the emotional state as the state s=(s e , s u ), where s e is the environmental parameter and s u is the emotional state; Define the environmental adjustment parameters of the smart home as the action a of reinforcement learning. The environmental adjustment parameters include, but are not limited to, air conditioner temperature, light brightness, and background music, denoted as a = (a 1 , a 2 , …, a Dn ), where Dn is the number of types of environmental adjustment parameters, and a Dn is the Dn-th type of environmental adjustment parameter; Calculate the amount of emotional improvement ΔE(s u , a) after the user takes action a, the adjustment cost C(a), and the user comfort U(s, a), and design a multi-objective reward function R(s, a); Introduce the state transition probability P(s’|s,a), use the reinforcement learning algorithm to learn the optimal environmental adjustment strategy, and the optimization goal is to maximize the expectation of the cumulative multi-objective reward function.

6. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 5, wherein, The method for calculating the emotional improvement amount ΔE(s u , a), regulation cost C(a), and user comfort U(s, a) after the user takes action a, and specifically designing the multi-objective reward function R(s, a) is as follows: According to the predicted probability p of the c-th emotion category in the current state c and the predicted probability p c ’ of the c-th emotion category after taking the action a, calculate the emotion improvement amount ΔE(s u , a); Calculate the adjustment cost C(a) of the environmental parameter according to the value of each environmental adjustment parameter a dn ; Obtain the user comfort U(s,a) of the user for the action a in the state s according to the user feedback; According to the amount of mood improvement ΔE(s u , a), the adjustment cost C(a), and the user comfort U(s, a), calculate the multi-objective reward function R(s, a).

7. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 6, characterized in that The specific method for judging the user's interaction intention and dynamically generating the interaction behavior decision of the smart home according to the degree and direction of the impact of the environmental adjustment strategy on the user's emotion based on the time series of the user's emotion characteristics collected in real time is as follows: Real-time collect the interaction action features between the user and the smart home. Let the duration of the g-th interaction action be Δt g , the force be f g , and the frequency be w g . Then the interaction action feature x of the g-th interaction action g = {Δt g , f g , w g}, where g = 1, 2,..., G, and G is the total number of interaction actions; Judge the user's interaction intention h based on the user's interaction action characteristics, use the random forest as a classification model, and use the interaction action characteristics as the input to predict the user's interaction intention h; Collect the time series of the user's emotion characteristics in real time before and after executing the environmental adjustment strategy. The time series of the emotion characteristics includes the pre-adjustment emotion sequence {ej(ta), ta = 1, 2,..., Ta} and the post-adjustment emotion sequence {ej(tb), tb = 1, 2,..., Tb}, and judge the degree and direction of the impact of the environmental adjustment strategy on the user's emotion; where ej(ta) and ej(tb) respectively represent the pre-adjustment emotion characteristics and the post-adjustment emotion characteristics, and Ta and Tb respectively represent the number of emotion characteristics in the pre-adjustment emotion sequence and the post-adjustment emotion sequence; Design the smart home interaction behavior decision rule based on the impact direction and the user's interaction intention, and generate the smart home interaction behavior U according to the smart home interaction behavior decision rule; Take the degree of the impact of the environmental adjustment strategy on the user's emotion as the execution intensity of the smart home interaction behavior, and output the interaction behavior and its execution intensity as the smart home interaction behavior decision.

8. The smart home AI ornament based on multi-sensor fusion emotion response according to claim 7, wherein, The specific method for judging the degree and direction of the impact of the environmental adjustment strategy on the user's emotion is as follows: Calculate the characteristic mean Ea of the pre-adjustment emotion sequence and the characteristic mean Eb of the post-adjustment emotion sequence respectively; Calculate the absolute value of the difference between the characteristic means of the pre-adjustment emotion sequence and the post-adjustment emotion sequence, which is used to represent the degree of the impact of the environmental adjustment strategy on the user's emotion Iab, and calculate the sign function of the difference, which is used to judge the direction of the impact of the environmental adjustment strategy on the user's emotion Dab. When Dab = 1, it represents a positive impact. When Dab = -1, it represents a negative impact. When Dab = 0, it represents no impact.

Images