Smart home ai figurine based on multi-sensor fusion emotional response

Abstract

The application relates to the technical field of artificial intelligence, and relates to an intelligent household AI ornament based on a multi-sensor fusion emotional response.The intelligent household AI ornament comprises an association graph construction module, an emotional model construction module and an environmental parameter adjustment module.The association graph construction module is used for collecting environmental parameters and emotional characteristics, calculating a cross correlation coefficient and performing Granger causality detection, and constructing an environmental emotional association graph.The emotional model construction module is used for converting the environmental emotional association graph into an adjacency matrix and a node feature matrix, inputting the adjacency matrix and the node feature matrix into an emotional data prediction model, and outputting a prediction probability of each emotional category.The environmental parameter adjustment module is used for learning an optimal environmental adjustment strategy by using a reinforcement learning algorithm.The interactive behavior decision module is used for judging an interactive intention of a user, judging an influence degree and an influence direction of the environmental adjustment strategy on the user's emotion according to a time sequence of the user's emotional characteristics collected in real time, and dynamically generating an interactive behavior decision of the intelligent household, so that self-adaptive adjustment of the intelligent household ornament is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of artificial intelligence, in particular to an intelligent home AI ornament based on multi-sensor fusion emotional response. BACKGROUND

[0002] Intelligent home refers to the use of advanced technology, such as the Internet, sensors, artificial intelligence, and other technologies, to achieve intelligent control and management of home devices, and to improve the quality of life and convenience.

[0003] Chinese patent application CN117492378A discloses an intelligent home regulation method and device based on emotion perception. The method includes determining the current emotional state of the user and the set of intelligent devices corresponding to the target area where the user is located, the set of intelligent devices including at least one intelligent device; analyzing the user's sensory adjustment information according to the current emotional state, the sensory adjustment information including adjustment information corresponding to at least one target sensory type; determining at least one target device that needs to be regulated and the regulation parameter corresponding to each target device according to the sensory adjustment information and the set of intelligent devices; for each target device, controlling the target device to perform the operation corresponding to the regulation parameter according to the regulation parameter corresponding to the target device.

[0004] The regulation method of existing emotion perception home devices usually only considers the user's emotion itself, ignoring the dynamic correlation and influence mechanism between environmental factors and user emotion. When the environmental parameters change, the emotional regulation strategy of intelligent home lacks environmental adaptability. SUMMARY

[0005] The present application aims to at least partially solve one of the technical problems in the related art. To this end, one object of the present application is to provide an intelligent home AI ornament based on multi-sensor fusion emotional response, which realizes adaptive adjustment of intelligent home ornaments.

[0006] One aspect of the present application provides an intelligent home AI ornament based on multi-sensor fusion emotional response, comprising:

[0007] A correlation graph construction module is configured to collect time series of environmental parameters and user emotional characteristics, calculate cross-correlation coefficients between environmental parameters and emotional characteristics, and perform Granger causality detection, and construct an environmental-emotional correlation graph based on the test results;

[0008] An emotional model construction module is configured to convert the environmental-emotional correlation graph into an adjacency matrix and a node feature matrix, input them into an emotional data prediction model, output prediction probabilities of each emotional category, and periodically update the emotional data prediction model;

[0009] An environmental parameter adjustment module is configured to define the environmental parameters and the predicted probabilities of each emotion category as states, define the environmental adjustment parameters of the smart home as actions, design a multi-objective reward function, and learn an optimal environmental adjustment strategy using a reinforcement learning algorithm.

[0010] An interactive behavior decision module is configured to determine the user's interactive intention, determine the influence degree and direction of the environmental adjustment strategy on the user's emotion based on the time series of the real-time collected user emotion features, and dynamically generate an interactive behavior decision of the smart home.

[0011] The specific method for collecting the time series of the environmental parameters and the user emotion features, calculating the cross-correlation coefficient between the environmental parameters and the emotion features, and performing Granger causality detection is as follows:

[0012] Step S110: integrating an environmental sensor on the smart home to collect environmental parameters in real time, wherein the environmental parameters include illumination, temperature, and humidity, and recording the environmental parameters in chronological order as an environmental parameter sequence wherein T1 is the length of the environmental parameter sequence, t1 is a variable representing a time point, t1={1, 2,..., T1}, represents the i-th environmental parameter at the time point t1;

[0013] Step S120: collecting user facial expression data through a camera and collecting user physiological signal data through a wearable device, extracting emotion features of the user from the facial expression data and the physiological signal data, and recording the emotion features in chronological order as an emotion feature sequence wherein T2 is the length of the emotion feature sequence, t2 is a variable representing a time point, t2={1, 2,..., T2}, represents the j-th emotion feature at the time point t2;

[0014] Step S130: time aligning and resampling the environmental parameter sequence and the emotion feature sequence to convert them into equal-length time sequences, respectively denoted as and wherein T' is the length of the aligned sequence, t is a variable representing a time point, t={1, 2,..., T'};

[0015] Step S140: obtaining an environmental parameter sub-sequence and an emotion feature sub-sequence from the environmental parameter sequence and the emotion feature sequence using a sliding window method, wherein a preset window length is w1 and a sliding step is s1, and obtaining the environmental parameter sub-sequence in the k1-th time window and the emotion feature sub-sequence

[0016] Step S150: calculating the cross-correlation coefficient r between the environmental parameter i and the emotion feature j for each sub-sequence in each time windowij (k1);

[0017] The specific method of constructing the environment emotion correlation graph based on the test result is:

[0018] Step S160: Based on the vector autoregressive model, a causal relationship model is constructed and trained, a Granger causality test is performed, and it is determined whether the environment parameter i is the Granger cause of the emotion feature j. If the coefficient γ jk of the environment parameter i is not equal to 0, it is considered that the environment parameter i is the Granger cause of the emotion feature j;

[0019] Step S170: Based on the result of the Granger causality test, an environment emotion correlation graph is constructed, the environment parameters and the emotion features are defined as nodes, the nodes include environment parameter nodes and emotion feature nodes, a relationship edge is established between the nodes with a cross-correlation coefficient greater than a correlation coefficient threshold, a directed edge is added according to the Granger cause. If node A' is the Granger cause of node B', a directed edge from node A' to node B' is added;

[0020] Step S180: The absolute value of the cross-correlation coefficient between the nodes is taken as the edge weight of the relationship edge;

[0021] The specific method of converting the environment emotion correlation graph into an adjacency matrix and a node feature matrix, inputting into an emotion data prediction model, outputting the prediction probability of each emotion category, and periodically updating the emotion data prediction model is:

[0022] Step S210: The environment emotion correlation graph is represented as an adjacency matrix JA and a node feature matrix JX. The element a ij of the adjacency matrix JA represents the edge weight w i between nodes v j and v ij , that is, a ij =w ij , if there is no edge between v i and v j , then a ij =0, and the element x ik of the node feature matrix JX represents the kth feature value of the node v i ;

[0023] Step S220: Based on the environment parameter subsequence and the emotion feature subsequence obtained by dividing the environment parameter sequence and the emotion feature sequence, the emotion category of the user at the end of each time window is taken as the emotion category label of the time window. The adjacency matrix and the node feature matrix of each time window are taken as the input, and the emotion category label of the next time window is taken as the output. Training samples are constructed;

[0024] Step S230: a depth map neural network model is constructed, the depth map neural network model comprises an input layer, a graph convolution layer and an output layer, an adjacency matrix JA and a node feature matrix JX are input into the input layer of the depth map neural network model;

[0025] Step S240: the graph convolution layer is composed of an L-layer convolutional neural network, for the l+1th layer, the node feature matrix and the adjacency matrix of the lth layer are taken as inputs, and the node feature matrix of the l+1th layer is output, and the graph convolution layer finally outputs the node feature matrix H of the Lth layer (L) , L is the number of network layers;

[0026] Step S250: the output layer maps the node feature matrix of the last layer to the predicted probability of each emotion category

[0027] Step S260: a cross-entropy loss function is taken as a loss function Loss of the depth map neural network model, the depth map neural network model is trained using a back propagation algorithm and an optimizer, a difference between the predicted probability and the real emotion category is calculated using the loss function, a value of the loss function is minimized, the training is completed when the loss function reaches convergence, and a trained emotion data prediction model is obtained;

[0028] Step S270: a sample buffer pool is designed, when a new training sample is collected, it is added to the sample buffer pool, and the earliest training sample is removed from the sample buffer pool, so that the size of the sample buffer pool is constant;

[0029] Step S280: a batch of training samples are randomly sampled from the sample buffer pool at regular intervals, and the emotion data prediction model is updated, a forgetting mechanism is designed, and a sample weight w of the training sample in the sample buffer pool is exponentially decayed with time t ;

[0030] The environment parameters, the predicted probability of each emotion category are defined as states, the environment adjustment parameters of the smart home are defined as actions, a multi-objective reward function is designed, and the specific method for learning the optimal environment adjustment strategy using a reinforcement learning algorithm is as follows:

[0031] Step S310: the user emotion category and the corresponding predicted probability are defined as the emotion state of the user, and the environment parameters and the emotion state are defined as the state s=(s e ,s u ) of the reinforcement learning, wherein s e is the environment parameter, and s u is the emotion state;

[0032] Step 320: define the environment adjustment parameters of the smart home as the actions a of reinforcement learning, which include but are not limited to air conditioner temperature, light brightness, background music, denoted as a=(a 1 ,a 2 ,…,a Dn ), wherein Dn is the number of types of environment adjustment parameters, and a Dn is the Dnth environment adjustment parameter;

[0033] Step S330: calculate the emotion improvement amount ΔE(s u ,a), the adjustment cost C(a), and the user comfort U(s,a) after the user takes the action a, and design a multi-objective reward function R(s,a);

[0034] The specific method for calculating the emotion improvement amount ΔE(s u ,a), the adjustment cost C(a), and the user comfort U(s,a) after the user takes the action a, and designing a multi-objective reward function R(s,a) is:

[0035] Step S331: calculate the emotion improvement amount ΔE(s c ,a) after the user takes the action a according to the prediction probability p c of the cth emotion category under the current state and the prediction probability p u ’ of the cth emotion category after taking the action a;

[0036] Step S332: calculate the adjustment cost C(a) of the environment parameter according to the value of each environment adjustment parameter a dn ;

[0037] Step S333: obtain the user comfort U(s,a) of the user to the action a under the state s according to the user feedback;

[0038] Step S334: calculate the multi-objective reward function R(s,a) according to the emotion improvement amount ΔE(s u ,a), the adjustment cost C(a), and the user comfort U(s,a);

[0039] Step S340: introduce the state transition probability P(s’|s,a), and use a reinforcement learning algorithm to learn the optimal environment adjustment strategy, with the optimization goal being to maximize the expectation of the cumulative multi-objective reward function;

[0040] The specific method for judging the interactive intention of the user, judging the influence degree and direction of the environment adjustment strategy on the user's emotion according to the time series of the real-time collected user emotion features, and dynamically generating the interactive behavior decision of the smart home is:

[0041] Step S410: Real-time collection of user interaction action features with smart home, and the gth interaction action duration is Δt g , the force is f g , and the frequency is w g , the gth interaction action feature x g = {Δt g , f g , w g}, g = 1, 2,..., G, and G is the total number of interaction actions;

[0042] Step S420: Judgment of user interaction intention h based on user interaction action features, and a random forest is used as a classification model to predict the user interaction intention h with the interaction action features as input;

[0043] Step S430: Real-time collection of time series of user emotional features before and after the execution of the environment adjustment strategy, the time series of emotional features including the pre-adjustment emotional sequence {ej(ta), ta = 1, 2,..., Ta} and the post-adjustment emotional sequence {ej(tb), tb = 1, 2,..., Tb}, and judgment of the influence degree and direction of the environment adjustment strategy on user emotions; wherein, ej(ta) and ej(tb) represent the pre-adjustment emotional features and the post-adjustment emotional features respectively, and Ta and Tb represent the number of emotional features in the pre-adjustment emotional sequence and the post-adjustment emotional sequence respectively;

[0044] Step S440: Design of smart home interaction behavior decision rules based on the influence direction and user interaction intention, and generation of smart home interaction behavior U according to the smart home interaction behavior decision rules;

[0045] Step S450: The influence degree of the environment adjustment strategy on user emotions is used as the execution force of the smart home interaction behavior, and the interaction behavior and its execution force are output as the smart home interaction behavior decision;

[0046] The acquisition method of the classification model is as follows:

[0047] Step S421: Acquisition of user interaction action features and corresponding interaction intention labeled by expert knowledge, normalization of the interaction action features, and conversion of the interaction intention into a one-hot encoding vector i h ;

[0048] Step S422: Formation of training samples from the user interaction action features and the interaction intention, obtaining of a data set, random sampling with replacement of a training sample subset D r from the data set, and random selection of an interaction action feature subset F r, and a single decision tree is constructed using the training sample subset and the interaction action feature subset;

[0049] Step S423: integrate the m single decision trees to define a random forest model as: wherein, represents the prediction result of the input X, h r (X) represents the predicted interaction intention category of the rth decision tree for the input X, represents the interaction intention category with the most votes;

[0050] The specific method for judging the influence degree and direction of the environmental adjustment strategy on the user emotion is:

[0051] Step S431: calculate the feature mean Ea of the emotion sequence before adjustment and the feature mean Eb of the emotion sequence after adjustment, respectively;

[0052] Step S432: calculate the absolute value of the difference between the feature means of the emotion sequence before adjustment and the emotion sequence after adjustment, which is used to represent the influence degree Iab of the environmental adjustment strategy on the user emotion, and calculate the sign function of the difference, which is used to judge the influence direction Dab of the environmental adjustment strategy on the user emotion. When Dab = 1, it represents positive influence; when Dab = -1, it represents negative influence; and when Dab = 0, it represents no influence.

[0053] The intelligent home AI ornament based on multi-sensor fusion emotion response proposed in the present application has the following advantages compared with the prior art:

[0054] The present application establishes the causal relationship between environmental parameters and emotion features by introducing an environmental emotion association graph, reveals the influence mechanism of the environment on the emotion, and based on the association graph, the emotion data prediction model can incorporate environmental factors and dynamically adapt to environmental changes, thereby improving the accuracy of emotion prediction when the environmental parameters change.

[0055] Based on the emotion prediction result and the current environmental parameter, the present application finds the optimal environmental adjustment strategy through a reinforcement learning algorithm, and uses the adjustment of the environmental parameter as a means to adjust the user emotion, so as to improve the user emotion while taking into account multiple goals, and realizes the emotion adjustment and collaborative optimization of the intelligent home ornament.

[0056] The interactive behavior decision of the present application comprehensively utilizes the environmental emotion association graph, the emotion prediction result and the environmental adjustment strategy, dynamically generates the most suitable interactive behavior by judging the influence of the environmental adjustment on the user emotion and combining the interactive intention. The interactive decision can actively adapt to the user emotion and environmental changes, and provide timely, accurate and personalized emotional interaction service.

[0057] The interactive behavior decision of the present application can provide positive and personalized emotional interaction services for users based on the influence of the environmental adjustment strategy on the user's emotions, and the environmental adjustment strategy is optimized based on the current environmental parameters and the prediction results of the user's emotions. The prediction model for predicting the user's emotions integrates environmental features and historical emotional features of the user and the influence mechanism between them, so that a dynamic closed-loop collaborative feedback optimization mechanism is formed among emotion prediction, environmental adjustment and interaction decision, providing continuous optimization of emotional response for users. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A functional module diagram of the smart home AI ornament based on multi-sensor fusion emotional response provided by the present application is provided.

[0059] Figure 2 A construction method flowchart of the environmental emotion association graph provided by the present application is provided.

[0060] Figure 3 A construction method flowchart of the multi-target reward function provided by the present application is provided.

[0061] Figure 4 A method flowchart of the reinforcement learning algorithm learning the optimal environmental adjustment strategy provided by the present application is provided.

[0062] Figure 5 A decision method flowchart of the smart home executing the interactive behavior decision provided by the present application is provided. DETAILED DESCRIPTION

[0063] In order to better understand the present application, various aspects of the present application will be described in more detail with reference to the accompanying drawings. It should be understood that these detailed descriptions are only descriptions of exemplary embodiments of the present application, and do not limit the scope of the present application in any way. Throughout the specification, the same reference numerals refer to the same elements. The expression "and / or" includes any and all combinations of one or more of the associated listed items.

[0064] In the drawings, the size, dimensions and shapes of the elements have been slightly adjusted for ease of illustration. The drawings are merely examples and are not strictly drawn to scale. As used in this document, the terms "approximately", "about", and similar terms are used as terms of approximation and not as terms of degree, and are intended to account for the inherent deviations in measured or calculated values that would be recognized by those of ordinary skill in the art. In addition, in the present application, the order of the steps of the process described does not necessarily represent the order in which the processes appear in actual operation, unless there is an explicit other limitation or it can be derived from the context.

[0065] It should also be understood that expressions such as "including", "including", "having", "containing", and / or "containing" and the like in the present specification are open and non-limiting expressions, which means that the presence of the stated features, elements and / or components is indicated, but the presence of one or more other features, elements, components and / or combinations thereof is not excluded. In addition, when expressions such as "at least one of" appear after a list of listed features, they modify the entire list of features, not just the individual elements in the list. In addition, when describing embodiments of the present application, "may" means "one or more embodiments of the present application". And the term "exemplary" is intended to refer to an example or illustration.

[0066] Unless otherwise defined, all terms (including engineering terms and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present application belongs. It should also be understood that, unless otherwise explicitly stated in the present application, words defined in common dictionaries should be interpreted to have meanings consistent with their meanings in the context of the relevant technology, and should not be interpreted in an idealized or overly formalized sense.

[0067] It should be noted that the embodiments and features in the present application can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in conjunction with the embodiments.

[0068] Embodiment 1

[0069] As Figure 1 shown, the smart home AI ornament based on multi-sensor fusion emotional response provided by the present application includes:

[0070] The correlation graph construction module is used to collect the time series of environmental parameters and user emotional characteristics, calculate the cross-correlation coefficient between the environmental parameters and the emotional characteristics, and perform Granger causality detection, and construct an environmental emotion correlation graph based on the test results;

[0071] The emotion model construction module is used to convert the environmental emotion correlation graph into an adjacency matrix and a node feature matrix, input into an emotional data prediction model, output the prediction probability of each emotional category, and periodically update the emotional data prediction model;

[0072] The environmental parameter adjustment module is used to define the environmental parameters and the prediction probability of each emotional category as a state, define the environmental adjustment parameters of the smart home as an action, design a multi-objective reward function, and learn the optimal environmental adjustment strategy using a reinforcement learning algorithm;

[0073] An interactive behavior decision module is configured to determine the interactive intention of the user, determine the influence degree and direction of the environmental adjustment strategy on the user emotion according to the time sequence of the real-time collected user emotion features, and dynamically generate the interactive behavior decision of the smart home.

[0074] The specific method for collecting the time sequence of the environmental parameters and the user emotion features, calculating the cross-correlation coefficient between the environmental parameters and the emotion features, and performing Granger causality detection is as follows:

[0075] Step S110: integrating an environmental sensor on the smart home to collect environmental parameters in real time, wherein the environmental parameters include illumination, temperature, and humidity, and recording the environmental parameters in time sequence as an environmental parameter sequence wherein T1 is the length of the environmental parameter sequence, t1 is a variable representing time, t1={1, 2,..., T1}, Xi(t1) represents the i-th environmental parameter at time t1;

[0076] Step S120: collecting user facial expression data through a camera and collecting user physiological signal data through a wearable device, extracting emotion features of the user from the facial expression data and the physiological signal data, and recording the emotion features in time sequence as an emotion feature sequence wherein T2 is the length of the emotion feature sequence, t2 is a variable representing time, t2={1, 2,..., T2}, Xj(t2) represents the j-th emotion feature at time t2;

[0077] The emotion features refer to the emotion category to which the user belongs and the degree of the emotion category.

[0078] Further, the collected environmental parameters, user facial expression data, and physiological signal data are transmitted to the smart home system through a wireless communication network, and the emotion features of the user are extracted from the user facial expression data and the physiological signal data.

[0079] Step S130: performing time alignment and resampling on the environmental parameter sequence and the emotion feature sequence to convert them into equal-length time sequences, respectively denoted as and wherein T' is the length of the aligned sequence, t is a variable representing time, t={1, 2,..., T'};

[0080] Step S140: obtaining environmental parameter subsequences and emotion feature subsequences from the environmental parameter sequence and the emotion feature sequence by using a sliding window method, wherein a preset window length is w1, a sliding step is s1, and the environmental parameter subsequence in the k1-th time window is and the emotion feature subsequence is

[0081] Step S150: for each sub-sequence within each time window, calculate the cross-correlation coefficient r between the environmental parameter i and the emotion feature j ij (k1) ;

[0082] The formula for calculating the cross-correlation coefficient between the environmental parameter i and the emotion feature j is: wherein, and are the mean values of the environmental parameter i and the emotion feature j within the k1th time window, respectively;

[0083] The specific method for constructing the environmental emotion correlation graph based on the test result is:

[0084] Step S160: based on the vector autoregressive model, construct and train a causal relationship model, perform Granger causality test to determine whether the environmental parameter i is the Granger cause of the emotion feature j, if the coefficient γ jk of the environmental parameter i is not equal to 0, it is considered that the environmental parameter i is the Granger cause of the emotion feature j;

[0085] The expression of the causal relationship model is: wherein, p is the lag order, a j is the constant term, β jk and γ jk are the coefficients of the emotion feature j and the environmental parameter i, respectively, is the emotion feature j at time t-k, is the environmental parameter i at time t-k, is a random disturbance term;

[0086] The causal relationship model is used to describe the dynamic relationship between variables, which indicates that the emotion feature is not only related to its own past value, but also related to the past value of the environmental parameter;

[0087] The principle of Granger causality test is: if variable 1 helps to understand the future change of variable 2, that is, adding the lag term of variable 1 to the model containing the past information of variable 2 itself can improve the prediction ability of variable 2, then variable 1 is called the Granger cause of variable 2;

[0088] The specific method for constructing and training the causal relationship model is:

[0089] Step S161: set the lag order p of the causal relationship model with the emotion feature as the endogenous variable and the environmental parameter as the exogenous variable, and construct the causal relationship model based on the vector autoregressive model;

[0090] The lag order p is set by the person skilled in the art according to experience;

[0091] Step S162: For each endogenous variable, an equation is formed by the expression of the causal relationship model, an equation group is obtained, the equation group is converted into a matrix form, and a standard form of the causal relationship model is obtained;

[0092] The standard form of the causal relationship model is Y t =A+B1×Y t-1 +...+B p ×Y t-p +C1×X t-1 +...+C p ×X t-p +E t , wherein Y t is the value of the m1 endogenous variables at time t, m1 represents the number of endogenous variables, A is a constant term of each endogenous variable, B1,..., B p are elements of an m1xm1 coefficient matrix, B p represents the influence coefficient of the endogenous variable when the endogenous variable lags p steps, Y t-p represents the value of the m1 endogenous variables at time t-p, C1,..., C p are elements of an m1xk1 coefficient matrix, C p represents the influence of the exogenous variable when the exogenous variable lags p steps, k1 is the number of exogenous variables, X t-p represents the value of the k1 exogenous variables at time t-p, and E t represents a random disturbance term vector;

[0093] Step S163: The environmental parameter sequence and the emotional feature sequence are taken as training data, the training data is used to train the causal relationship model, and the coefficients of the causal relationship model are estimated by a maximum likelihood estimation method;

[0094] The coefficients of the causal relationship model include the constant term of the endogenous variable, the m1xm1 coefficient matrix, and the m1xk1 coefficient matrix;

[0095] Step S164: The coefficients of the causal relationship model are subjected to significance detection, it is judged which coefficients are not 0, the coefficients corresponding to the non-0 coefficients are taken as lag terms and are included in the causal relationship model, and an estimated causal relationship model is obtained;

[0096] Step S170: Based on the result of the Granger causality test, an environmental emotion correlation graph is constructed, the environmental parameters and the emotional features are defined as nodes, the nodes include environmental parameter nodes and emotional feature nodes, a relationship edge is established between the nodes whose cross-correlation coefficient is greater than a correlation coefficient threshold, a directed edge is added according to the Granger cause, and if node A' is the Granger cause of node B', a directed edge from node A' to node B' is added;

[0097] The attributes of the environmental parameter nodes include statistical quantities of the respective environmental parameters, such as mean and variance;

[0098] The attributes of the emotion feature nodes include frequencies or intensities of the respective emotion categories;

[0099] The correlation coefficient threshold is set by a person skilled in the art according to experience;

[0100] Step S180: Taking the absolute value of the cross-correlation coefficient between the nodes as the edge weight of the relationship edge;

[0101] Figure 2 The environmental emotion correlation graph provided in the present application is constructed by the method flowchart;

[0102] The above steps introduce the environmental emotion correlation graph, establish the causality and correlation between the environmental parameters and the user emotions, reveal the dynamic influence mechanism of the environmental factors on the user emotions, and provide key decision-making basis for emotion prediction and intelligent regulation.

[0103] The specific method of converting the environmental emotion correlation graph into an adjacency matrix and a node feature matrix, inputting the adjacency matrix and the node feature matrix into the emotion data prediction model, outputting the prediction probability of each emotion category, and periodically updating the emotion data prediction model is as follows:

[0104] Step S210: Representing the environmental emotion correlation graph as an adjacency matrix JA and a node feature matrix JX, the element a ij of the adjacency matrix JA represents the edge weight w i between the nodes v j and v ij , that is, a ij =w ij , if there is no edge connected between the nodes v i and v j , then a ij =0, and the element x ik of the node feature matrix JX represents the kth feature value of the node v i ;

[0105] The row of the node feature matrix represents the feature of a node, and the column represents the node feature dimension;

[0106] Step S220: Taking the emotion category of the user at the end of each time window as the emotion category label of the time window, taking the adjacency matrix and the node feature matrix of each time window as the input, and taking the emotion category label of the next time window as the output, constructing a training sample based on the environmental parameter subsequence and the emotion feature subsequence obtained by dividing the environmental parameter sequence and the emotion feature sequence;

[0107] The adjacency matrix and the node feature matrix of each time window are updated based on the environmental parameters and the emotion features of the time window, and are taken as inputs again;

[0108] Step S230: constructing a deep graph neural network model, the deep graph neural network model comprising an input layer, a graph convolution layer, and an output layer, the adjacency matrix JA and the node feature matrix JX being input into the input layer of the deep graph neural network model;

[0109] Step S240: the graph convolution layer being composed of L layers of convolutional neural networks, for the l+1th layer, taking the node feature matrix and the adjacency matrix of the lth layer as inputs, outputting the node feature matrix of the l+1th layer, and the graph convolution layer finally outputting the node feature matrix H (L) of the Lth layer, L being the number of network layers;

[0110] The node feature matrix of the l+1th layer is represented as: wherein H (l+1) and H (l) are the node feature matrices of the l+1th and lth layers respectively, σ(·) is an activation function, W (l) is a weight matrix of the lth layer, is an adjacency matrix with self-connection added, is a degree matrix of ;

[0111] Step S250: the output layer mapping the node feature matrix of the last layer to a predicted probability of each emotion category

[0112] The predicted probability of the emotion category is: wherein f(·) is a softmax activation function, W o is a weight matrix of the output layer, and b o is a bias term;

[0113] The predicted probability of the emotion category is a probability value of the user emotion belonging to each emotion category;

[0114] Step S260: taking a cross-entropy loss function as a loss function Loss of the deep graph neural network model, training the deep graph neural network model using a back propagation algorithm and an optimizer, calculating the difference between the predicted probability and the real emotion category using the loss function, minimizing the value of the loss function, and completing the training when the loss function reaches convergence, to obtain a trained emotion data prediction model;

[0115] Step S270: design a sample buffer pool, when a new training sample is collected, add it to the sample buffer pool, and remove the earliest training sample from the sample buffer pool, keep the size of the sample buffer pool constant;

[0116] Step S280: periodically sample a batch of training samples from the sample buffer pool, update the emotion data prediction model, design a forgetting mechanism, and assign an exponentially decaying sample weight w t to the training samples in the sample buffer pool over time

[0117] The calculation formula of the exponentially decaying sample weight w t is as follows: Where w t is the sample weight at time t, t0 is the time when the training sample enters the sample buffer pool, and λ is the decay factor

[0118] The specific value of the decay factor is set by a person skilled in the art according to experience, λ∈(0,1)

[0119] The batch size of the batch of randomly sampled training samples is set by a person skilled in the art according to experience

[0120] Further, in steps S270 and S280, the related data of the training samples in the sample buffer pool are uploaded to the cloud server through the wireless communication network, the emotion data prediction model is updated on the cloud server, and the updated model parameters are downloaded to the local device.

[0121] The above steps convert the environmental emotion association graph into an adjacency matrix and a node feature matrix, input them into a deep graph neural network model for emotion prediction, and design a forgetting mechanism for periodic updating of the model, realizing end-to-end emotion prediction based on graph structure data, and significantly improving the accuracy and timeliness of the prediction.

[0122] The specific method for defining the environment parameters and the prediction probabilities of each emotion category as states, defining the environment adjustment parameters of the smart home as actions, and designing a multi-objective reward function to learn the optimal environment adjustment strategy using a reinforcement learning algorithm is as follows:

[0123] Step S310: define the user emotion category and the corresponding prediction probability as the emotional state of the user, and define the environment parameters and the emotional state as the state of reinforcement learning s=(s e ,s u ), where s e is the environment parameter, and s u is the emotional state

[0124] Step 320: define the environment adjustment parameters of the smart home as the actions a of reinforcement learning, including but not limited to air conditioner temperature, light brightness, background music, denoted as a=(a 1 ,a 2 ,…,a Dn ), wherein Dn is the number of types of environment adjustment parameters, a Dn is the Dnth environment adjustment parameter;

[0125] The execution of the action of the reinforcement learning controls the smart home ornaments through a wireless communication network;

[0126] Step S330: calculate the emotion improvement amount ΔE(s u ,a), the adjustment cost C(a), and the user comfort U(s,a) after the user takes the action a, and design a multi-objective reward function;

[0127] The specific method for calculating the emotion improvement amount ΔE(s u ,a), the adjustment cost C(a), and the user comfort U(s,a) after the user takes the action a, and designing a multi-objective reward function is:

[0128] Step S331: calculate the emotion improvement amount ΔE(s c ,a) after the user takes the action a according to the prediction probability p c of the cth emotion category under the current state and the prediction probability p u ’ of the cth emotion category after taking the action a;

[0129] The calculation formula of the emotion improvement amount is: wherein w c is the weight coefficient of the cth emotion category, and C is the total number of emotion categories;

[0130] The weight coefficient of the emotion category is set by a person skilled in the art according to experience;

[0131] Step S332: calculate the adjustment cost C(a) of the environment parameter according to the value of each environment adjustment parameter a dn ;

[0132] The calculation formula of the adjustment cost is: wherein a dn is the cost coefficient of the dnth environment adjustment parameter, and a dn is the dnth environment adjustment parameter;

[0133] The cost coefficient of the environment adjustment parameter is set by a person skilled in the art according to experience;

[0134] Step S333: obtaining the user comfort U(s, a) of the user to the action a in the state s according to the user feedback;

[0135] The user comfort U(s, a) has a value range of [0, 1];

[0136] Step S334: calculating a multi-objective reward function according to the emotion improvement amount ΔE(s u ,a), the adjustment cost C(a), and the user comfort U(s, a);

[0137] The calculation formula of the multi-objective reward function is: Wherein, β1, β2, β3, and β4 are weight coefficients of the balance emotion improvement amount, the adjustment cost, the user comfort, and the interaction item of the emotion improvement amount and the user comfort, λ1 and λ2 are parameters for controlling the steepness of the Sigmoid function, μ1 and μ2 are midpoint parameters of the Sigmoid function, and γ is an exponential parameter of the adjustment cost.

[0138] The exponential parameter of the adjustment cost is used to control the nonlinearity degree of the cost, and the weight coefficients of the emotion improvement amount, the adjustment cost, the user comfort, and the interaction item of the emotion improvement amount and the user comfort, the exponential parameter of the adjustment cost, and the midpoint parameters of the Sigmoid function are set by the person skilled in the art according to experience.

[0139] Figure 3 The construction method flowchart of the multi-objective reward function provided in the present application is shown in the following figure:

[0140] The above multi-objective reward function performs Sigmoid transformation on the emotion improvement amount and the user comfort, and can reflect their diminishing marginal utility characteristics, that is, when the improvement amount or the comfort is already high, the marginal effect of further improvement will decrease.

[0141] The adjustment cost adopts an exponential form, which can reflect the non-linear rapid growth of the cost.

[0142] The interaction item of the emotion improvement amount and the comfort is added, indicating that there is a positive correlation between them, that is, the more the emotion is improved, the more comfortable the user is likely to be.

[0143] Step S340: introducing a state transition probability P(s’|s, a), and using a reinforcement learning algorithm to learn an optimal environment adjustment strategy, and the optimization goal is to maximize the expectation of the cumulative multi-objective reward function;

[0144] The expectation of the cumulative multi-objective reward function is: Wherein, Q(s, a) is the expectation of the cumulative multi-objective reward function of taking the action a in the state s, and γ Qis a discount factor, s' is the next state, a' is the next action, is the maximum Q-value of the next state, Q(s',a') is the expectation of the cumulative multi-objective reward function of taking action a' in state s', represents the expectation of the next state s' after taking action a in state s;

[0145] The state transition probability represents the probability of transitioning to state s' after taking action a in state s;

[0146] Figure 4 The method flow chart for learning the optimal environment regulation strategy provided by the reinforcement learning algorithm of the present application;

[0147] The above steps take the environmental parameters and the emotional prediction results as the state, take the environmental regulation parameters as the action, design a multi-objective reward function, and learn the optimal environment regulation strategy by using a reinforcement learning algorithm, thereby realizing adaptive optimization of environment regulation, improving user emotions while taking into account multiple objectives such as energy consumption and comfort.

[0148] The specific method for judging the user's interactive intention is to judge the influence degree and influence direction of the environment regulation strategy on the user's emotions according to the time series of the real-time collected user emotional features, and dynamically generate the interactive behavior decision of the smart home.

[0149] Step S410: Real-time collection of user interactive action features of the smart home g , the force is f g , and the frequency is w g , then the interactive action features x g of the gth interactive action are g , f g , and w g , g=1,2,...,G, and G is the total number of interactive actions;

[0150] Step S420: Judging the user's interactive intention h based on the user's interactive action features, using a random forest as a classification model, taking the interactive action features as input, and predicting the user's interactive intention h;

[0151] The acquisition method of the classification model is:

[0152] Step S421: Obtaining the user's interactive action features and the corresponding interactive intention labeled by expert knowledge, normalizing the interactive action features, and converting the interactive intention into a one-hot encoding vector i h ;

[0153] The one-hot encoding vector i h is represented as: ih = [I(i h = 1), I(i h = 2),..., I(i h = M)], where i h is a one-hot encoding vector of the interactive intention h, M is the number of categories of the interactive intention, i h = M indicates that the interactive intention category of the user is the Mth interactive intention, I(·) is an indicator function, which takes the value of 1 when the condition is true, and 0 otherwise;

[0154] Step S422: Form a training sample from the interactive action features and the interactive intention of the user, obtain a data set, and randomly draw a training sample subset D r from the data set with replacement, randomly select an interactive action feature subset F r from all interactive action features, and use the training sample subset and the interactive action feature subset to build a single decision tree;

[0155] Step S423: Integrate mr decision trees from the single decision tree to define a random forest model as: wherein, represents the prediction result of the input X, h r (X) represents the predicted interactive intention category of the rth decision tree for the input X, represents the interactive intention category with the most votes;

[0156] Step S430: Real-time collect the time series of the emotional features of the user before and after executing the environmental adjustment strategy, the time series of the emotional features include the pre-adjustment emotional sequence {ej(ta), ta=1, 2,..., Ta} and the post-adjustment emotional sequence {ej(tb), tb=1, 2,..., Tb}, and determine the influence degree and direction of the environmental adjustment strategy on the user's emotion; wherein, ej(ta), ej(tb) respectively represent the pre-adjustment emotional features and the post-adjustment emotional features, Ta, Tb respectively represent the number of emotional features in the pre-adjustment emotional sequence and the post-adjustment emotional sequence;

[0157] The specific method for determining the influence degree and direction of the environmental adjustment strategy on the user's emotion is:

[0158] Step S431: Calculate the feature mean Ea of the pre-adjustment emotional sequence and the feature mean Eb of the post-adjustment emotional sequence, respectively;

[0159] The calculation formula of the feature mean Ea of the pre-adjustment emotional sequence is:

[0160] The calculation formula of the feature mean Eb of the post-adjustment emotional sequence is:

[0161] Step S432: Calculate the absolute value of the difference between the mean values of the features of the pre-adjustment emotion sequence and the post-adjustment emotion sequence, to represent the influence degree Iab of the environmental adjustment strategy on the user's emotion, and calculate the sign function of the difference, to determine the influence direction Dab of the environmental adjustment strategy on the user's emotion, wherein when Dab = 1, it represents positive influence, when Dab = -1, it represents negative influence, and when Dab = 0, it represents no influence;

[0162] The calculation formula of the influence degree of the environmental adjustment strategy on the user's emotion is Iab = |Ea-Eb|;

[0163] The calculation formula of the influence direction of the environmental adjustment strategy on the user's emotion is Dab = sign(Ea-Eb);

[0164] The value of the influence direction Dab is {1, 0, -1}, when Ea-Eb>0, sign(Ea-Eb)=1; when Ea-Eb<0, sign(Ea-Eb)=-1; when Ea-Eb=0, sign(Ea-Eb)=0;

[0165] Step S440: Design an intelligent home interaction behavior decision rule based on the influence direction and the user's interaction intention, and generate an interaction behavior U of the intelligent home according to the intelligent home interaction behavior decision rule;

[0166] The intelligent home interaction behavior decision rule is:

[0167] When Dab = 1 and h = pacification, U = rub hands;

[0168] When Dab = -1 and h = pacification, U = lie down;

[0169] When Dab = 1 and h = feeding, U = rub legs;

[0170] When Dab = -1 and h = feeding, U = sit down;

[0171] When Dab = 1 and h = play, U = bark;

[0172] When Dab = -1 and h = play, U = wag tail;

[0173] The intelligent home is a simulated pacification dog, which integrates various sensors for obtaining environmental parameters and user facial expression data.

[0174] Step S450: taking the influence degree of the environment adjustment strategy on the user emotion as the execution strength of the interactive behavior of the smart home, outputting the interactive behavior and the execution strength as the interactive behavior decision of the smart home;

[0175] The interactive behavior decision of the smart home collects the interactive intention and emotion characteristics of the user through a wireless communication network connecting the smart home, and the smart home executes the corresponding interactive behavior decision according to the interactive intention and emotion characteristics;

[0176] As shown in the above Figure 5 The decision method flow chart of the smart home executing the interactive behavior decision provided by the present application is shown in the above

[0177] The above steps dynamically generate personalized interactive behavior decisions by real-time perception of user interactive behavior, judgment of interactive intention, and analysis of the influence of environment adjustment on emotion combined with time series analysis of emotion characteristics. The smart home integrates the perception, adjustment and mutual feedback of environment and emotion, forms a closed loop with the foregoing steps, and can provide continuous and personalized emotional assistance services.

[0178] In addition, the parts of the above technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive repetition.

[0179] The specific embodiments described above further detail the purposes, technical solutions and beneficial effects of the present application. It should be understood that the above description is only a specific embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent home AI figurine based on multi-sensor fusion emotional response, characterized by, include: The association graph construction module is used to collect time series of environmental parameters and user emotional characteristics, calculate the cross-correlation coefficient between environmental parameters and emotional characteristics, perform Granger causality detection, and construct an environmental-emotion association graph based on the test results. The emotion model building module is used to transform the environmental emotion association graph into an adjacency matrix and a node feature matrix, input them into the emotion data prediction model, output the predicted probability of each emotion category, and update the emotion data prediction model regularly. The environmental parameter adjustment module is used to define environmental parameters and the predicted probabilities of each emotion category as states, define the environmental adjustment parameters of smart homes as actions, design a multi-objective reward function, and use reinforcement learning algorithms to learn the optimal environmental adjustment strategy. The interactive behavior decision module is used to determine the user's interaction intention, and to determine the degree and direction of the influence of the environmental adjustment strategy on the user's emotions based on the time series of user emotional characteristics collected in real time, and to dynamically generate interactive behavior decisions for smart homes; the time series of user emotional characteristics are collected in real time before and after the execution of the environmental adjustment strategy, and the time series of emotional characteristics includes the emotional sequence before adjustment and the emotional sequence after adjustment, to determine the degree and direction of the influence of the environmental adjustment strategy on the user's emotions. The specific method for judging the influence degree and influence direction of the environment adjustment strategy on the user emotion is: calculating the feature mean Ea of the emotion sequence before adjustment and the feature mean Eb of the emotion sequence after adjustment respectively; calculating the absolute value of the difference between the feature means of the emotion sequence before adjustment and the emotion sequence after adjustment, which is used to represent the influence degree of the environment adjustment strategy on the user emotion , calculating the sign function of the difference, which is used to judge the influence direction of the environment adjustment strategy on the user emotion , when , it represents positive influence, when , it represents negative influence, and when , it represents no influence; The specific method for constructing an environmental sentiment association map based on the test results is as follows: Based on the vector autoregressive model, a causal relationship model is constructed and trained, and a Granger causality test is performed to determine whether the environmental parameters are Granger causes of the emotional features . If the coefficient of the environmental parameters is not equal to 0, it is considered that the environmental parameters are Granger causes of the emotional features . The specific method for constructing and training the causal relationship model is as follows: Emotional characteristics are used as endogenous variables and environmental parameters are used as exogenous variables. The lag order p of the causal relationship model is set, and the causal relationship model is constructed based on the vector autoregression model. For each endogenous variable, an equation is formed by the expression of the causal relationship model, and a system of equations is obtained. The system of equations is transformed into matrix form to obtain the standard form of the causal relationship model. Using environmental parameter sequences and emotional feature sequences as training data, the causal relationship model is trained using the training data, and the coefficients of the causal relationship model are estimated using the maximum likelihood estimation method. Significance tests are performed on the coefficients of the causal relationship model to determine which coefficients are not zero. The terms corresponding to the non-zero coefficients are included as lag terms in the causal relationship model to obtain the estimated causal relationship model. Based on the results of the Granger causality test, an environmental-emotion correlation graph is constructed. Environmental parameters and emotional features are defined as nodes, including environmental parameter nodes and emotional feature nodes. Relationship edges are established between nodes with cross-correlation coefficients greater than a correlation coefficient threshold. Directed edges are added according to Granger causes. It is a node If the Granger cause is to be added, then add a node. Pointing to node The directed edges; The absolute value of the cross-correlation coefficient between nodes is used as the edge weight of the relation edge. 2.The smart home AI ornament based on multi-sensor fusion emotional response of claim 1, wherein The specific method for collecting time series data of environmental parameters and user emotional characteristics, calculating the cross-correlation coefficient between environmental parameters and emotional characteristics, and performing Granger causality detection is as follows: Integrate the environment sensor on the smart home, collect the environment parameter in real time, the environment parameter includes illumination, temperature, humidity, record as the environment parameter sequence according to time sequence Wherein, T1 is the length of the environment parameter sequence, t1 is a variable, represents the time, t1={1,2,...,T1}, The i-th environment parameter at the time t1; The facial expression data of the user is collected through a camera, physiological signal data of the user is collected through a wearable device, emotion features of the user are extracted from the facial expression data and the physiological signal data, and the emotion features are recorded as an emotion feature sequence in chronological order wherein T2 is a length of the emotion feature sequence, t2 is a variable representing a time, t2={1, 2, …, T2}, the jth emotion feature at the time t2. The environmental parameter sequence and the emotion feature sequence are time-aligned and resampled to be converted into equal-length time sequences, respectively denoted as and wherein is the length of the aligned sequence, t is a variable representing a time, t={1,2,..., }. The environment parameter subsequence and the emotion feature subsequence are obtained according to the environment parameter sequence and the emotion feature sequence by using a sliding window method, a preset window length is 1, and a sliding step is 1. 1, to obtain an environment parameter subsequence and an emotion feature subsequence in a first time window. For each sub-sequence within a time window, a cross-correlation coefficient between the environmental parameters and the emotional features is computed . 3.The smart home AI ornament based on multi-sensor fusion emotional response of claim 2, wherein, The specific method for converting the environmental emotion association graph into an adjacency matrix and a node feature matrix, inputting them into the emotion data prediction model, outputting the predicted probability of each emotion category, and periodically updating the emotion data prediction model is as follows: Representing the environmental-emotional association graph as an adjacency matrix and node feature matrix Adjacency matrix elements Represents a node and Edge weights between ,Right now ,like and If there are no edges connecting them, then Node feature matrix elements Represents a node The One eigenvalue; Based on the environmental parameter subsequence and emotional feature subsequence obtained by dividing the environmental parameter sequence and emotional feature sequence, the emotional category of the user at the end of each time window is used as the emotional category label of that time window. The adjacency matrix and node feature matrix of each time window are used as inputs, and the emotional category label of the next time window is used as the output to construct training samples. constructing a depth map neural network model, the depth map neural network model comprising an input layer, a graph convolutional layer, an output layer, inputting an adjacency matrix and a node feature matrix to the input layer of the depth map neural network model; The graph convolution layer is composed of an L-layer convolutional neural network. For the l+1th layer, the node feature matrix and the adjacency matrix of the lth layer are taken as inputs, and the node feature matrix of the l+1th layer is output. The graph convolution layer finally outputs the node feature matrix of the Lth layer L is the number of network layers. The output layer maps the node feature matrix of the last layer to the prediction probability of each emotion category ; The cross-entropy loss function is used as a loss function Loss of the depth map neural network model, the depth map neural network model is trained using a back propagation algorithm and an optimizer, the difference between the predicted probability and the true emotion category is calculated using the loss function, the value of the loss function is minimized, the training is completed when the loss function converges, and a trained emotion data prediction model is obtained; A sample buffer pool is designed, when a new training sample is collected, it is added to the sample buffer pool, and the oldest training sample is removed from the sample buffer pool, so that the size of the sample buffer pool is constant; Periodically, a batch of training samples is randomly sampled from the sample buffer pool, and the emotion data prediction model is updated, a forgetting mechanism is designed, and the training samples in the sample buffer pool are given sample weights that decay exponentially over time . 4.The smart home AI ornament based on multi-sensor fusion emotional response of claim 3, wherein The environmental parameters and the predicted probabilities of each emotion category are defined as states, the environmental adjustment parameters of the smart home are defined as actions, a multi-objective reward function is designed, and the specific method for learning the optimal environmental adjustment strategy using a reinforcement learning algorithm is: The user emotion category and the corresponding prediction probability are defined as the emotional state of the user, and the environmental parameters and the emotional state are defined as the state of reinforcement learning wherein, is the environmental parameter, is the emotional state; The environmental regulation parameters of the smart home are defined as the actions a of reinforcement learning, including but not limited to air conditioner temperature, light brightness, background music, denoted as wherein Dn is the number of types of environmental regulation parameters, is the Dnth environmental regulation parameter; computing an amount of improvement in mood of a user after taking action a , regulation cost , user comfort , designing a multi-objective reward function ; Introducing state transition probabilities An optimal environment conditioning policy is learned using a reinforcement learning algorithm, with the optimization goal being to maximize the expectation of a cumulative multi-objective reward function. 5.The smart home AI ornament based on multi-sensor fusion emotional response of claim 4, wherein, the amount of improvement in the user's mood after the user takes action a , regulation cost , user comfort , design a multi-objective reward function The specific method is: a predicted probability of the cth emotion category given the current state a predicted probability of the cth emotion category given the action a was taken , compute an amount of improvement in the user's emotion given the action a was taken ; According to the value of each environmental regulation parameter the adjustment cost of the environmental parameter is calculated ; user comfort level of the user for the action a in state s based on user feedback ; According to the amount of mood improvement , adjust the cost , user comfort , calculate the multi-objective reward function . 6.The smart home AI ornament based on multi-sensor fusion emotional response of claim 5, wherein, The specific method for judging the user's interaction intention according to the time series of the real-time collected user emotion features to judge the influence degree and direction of the environmental adjustment strategy on the user's emotion and dynamically generating the interaction behavior decision of the smart home is: Real-time collection of user interaction action features with smart home, set the gth interaction action duration as , force as , frequency as , then the gth interaction action feature , g=1,2,...,G, G is the total number of interaction actions; The user's interaction intention h is judged based on the user's interaction action features, a random forest is used as a classification model, and the interaction action features are used as input to predict the user's interaction intention h; The time series of the emotional features comprises a pre-regulation emotional sequence and a post-regulation emotional sequence ; wherein, , respectively represent the pre-regulation emotional features and the post-regulation emotional features, , respectively represent the number of emotional features in the pre-regulation emotional sequence and the post-regulation emotional sequence. The smart home interaction behavior decision rule based on the influence direction and the user's interaction intention is designed, and the smart home interaction behavior U is generated according to the smart home interaction behavior decision rule; The influence degree of the environmental adjustment strategy on the user's emotion is used as the execution degree of the smart home interaction behavior, and the interaction behavior and its execution degree are output as the interaction behavior decision of the smart home.

Images