Intelligent product interaction experience improvement method based on emotion recognition driving

Abstract

This invention relates to the field of intelligent product interaction, specifically disclosing a method for improving the interactive experience of intelligent products based on emotion recognition, including the following steps: a: continuously collecting users' physiological data, voice information, facial expressions, and body language through collection devices, including cameras, microphones, and biometric sensors; b: preprocessing the collected data to remove noise and extract key features to obtain a set of feature vectors. This invention collects user emotional data in real time, uses AI algorithms to analyze and identify the user's emotional state, and then adjusts the interaction method of intelligent products to provide more personalized and humanized services. This not only enhances the emotional connection between users and products, improving user satisfaction and loyalty, but also improves the user experience by increasing interaction efficiency and adaptive services.

Description

Technical Field

[0001] This invention belongs to the field of intelligent product interaction, specifically a method for improving the interactive experience of intelligent products based on emotion recognition. Background Technology

[0002] Intelligent product interaction refers to the use of advanced technologies to enable intelligent products to interact with users efficiently, intuitively, and in accordance with users' emotional needs. Through voice recognition technology, users can interact with intelligent products using natural language, such as checking the weather or playing music. Using touchscreen technology, users can interact with intelligent products by touching, swiping, and clicking. By displaying information on a screen, users can receive information visually and interact with the product.

[0003] Existing smart products often lack emotional understanding capabilities and cannot adjust their interaction methods according to the user's emotional state. As a result, smart products cannot understand the user's emotional needs, leading to poor overall user satisfaction and loyalty. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method for enhancing the interactive experience of intelligent products based on emotion recognition, thereby solving the problems that existing intelligent products often lack emotional understanding capabilities and cannot adjust their interaction methods according to the user's emotional state.

[0005] The method for improving the interactive experience of intelligent products based on emotion recognition includes the following steps:

[0006] a: By collecting devices, including cameras, microphones, and biometric sensors, we continuously collect users' physiological data, voice information, facial expressions, and body language;

[0007] b: The collected data is preprocessed to remove noise and extract key features to obtain a set of feature vectors;

[0008] c: Utilize a pre-trained AI-based emotion recognition algorithm and incorporate a feature fusion mechanism to fuse features from different collection devices;

[0009] d: Based on the fused feature vectors, the emotion recognition algorithm is used to analyze the user's emotional state, including happiness, sadness, anger, surprise, fear, and disgust;

[0010] e: Based on the identified emotional state, adjust the interaction method of smart products in real time to provide an experience that better matches the user's current emotion.

[0011] Preferably, the feature fusion mechanism uses the following mathematical formula when fusing features from different collection devices:

[0012]

[0013] Where, F i The feature vector extracted from the i-th sensor, α i It is the weight coefficient corresponding to the i-th feature vector, used to adjust the importance of different features during feature fusion.

[0014] Preferably, convolutional neural networks and recurrent neural networks are used, with long short-term memory networks to capture complex emotional expressions and time dependencies;

[0015] In deep learning algorithms, convolutional layers are used to process the feature vector set F, extracting higher-level features through convolution operations. The mathematical expression for convolution operation is:

[0016]

[0017] Where Y is the output of the convolution operation, F is the input feature map, W is the convolution kernel, (i,j) is the position index on the output feature map, and (m,n) is the size of the convolution kernel;

[0018] After the convolutional layer, the ReLU activation function is applied to increase the nonlinearity of the model. The mathematical expression for ReLU is:

[0019] ReLU(x) = max(0,x)

[0020] Pooling layers are used to reduce the dimensionality of feature maps while preserving important feature information. The mathematical expression for max pooling is:

[0021]

[0022] Where pool_window is the size of the pooling window;

[0023] After multiple convolutional layers, activation functions, and pooling layers, a fully connected layer is used to map the features to the sentiment label space. The output of the fully connected layer is converted into a probability distribution by the Softmax function, representing the confidence level of different sentiment labels. The mathematical expression is as follows:

[0024] O = Softmax(W·Z + b)

[0025] Where O is the output of emotion recognition, W is the weight matrix of the fully connected layer, Z is the output of the last convolutional layer, and b is the bias term;

[0026] Based on the output O of emotion recognition, the interaction mode of smart products is adjusted in real time to provide an experience that is more in line with the user's current emotion.

[0027] Preferably, the training of the emotion recognition algorithm includes:

[0028] a: Collect a large amount of sample data with sentiment labels, which comes from public datasets and data collected specifically for this purpose;

[0029] b: By using supervised learning, these data are used to train the emotion recognition algorithm to improve its accuracy in recognizing different emotional states;

[0030] c: Regularly update and optimize the algorithm to adapt to the emotional changes of different user groups and scenarios.

[0031] Preferably, the emotion recognition algorithm also considers contextual information, such as the user's historical behavior, current environment, and time, to more accurately identify the user's emotional state.

[0032] Preferably, adjusting the interaction methods of smart products includes:

[0033] a: Change the colors, brightness, icons, and layout of the display interface to match the user's emotional state;

[0034] b: Adjust the tone, speed, and pitch of voice interaction to better suit the user's emotional needs;

[0035] c: Provide content and suggestions related to your current emotional state, such as recommending music, movies, and articles;

[0036] d: When negative emotions are detected in a user, specific soothing mechanisms are activated, including playing soothing music, providing links to emotional counseling, and offering deep breathing suggestions.

[0037] Preferably, it also includes establishing user emotional profiles to record users' emotional states at different times and in different scenarios, in order to analyze the trend of user emotional changes and provide data support for subsequent interaction optimization.

[0038] Preferably, smart products also have self-learning and optimization capabilities, and can adjust emotion recognition algorithms and interaction strategies based on user feedback and real-time data to continuously improve the user experience.

[0039] Preferably, the smart product supports collaborative work with other smart devices, enabling cross-device emotional interaction experiences by sharing emotion recognition results and interaction strategies.

[0040] Compared with existing technologies, the present invention has the following beneficial effects: by collecting user emotional data in real time, using AI algorithms to analyze and identify the user's emotional state, and then adjusting the interaction mode of smart products to provide more personalized and humanized services, this not only enhances the emotional connection between users and products, improving user satisfaction and loyalty, but also improves user experience by increasing interaction efficiency and adaptive services. In addition, this method provides new directions for product innovation and functional expansion, enhancing the product's market competitiveness and sustainable development capabilities. Attached Figure Description

[0041] Figure 1 This is a schematic diagram of the method of the present invention. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] like Figure 1 As shown:

[0044] Example 1: This invention provides a method for improving the interactive experience of intelligent products based on emotion recognition, comprising the following steps:

[0045] a: By collecting devices, including cameras, microphones, and biometric sensors, we continuously collect users' physiological data, voice information, facial expressions, and body language;

[0046] b: The collected data is preprocessed to remove noise and extract key features to obtain a set of feature vectors;

[0047] c: Utilize a pre-trained AI-based emotion recognition algorithm and incorporate a feature fusion mechanism to fuse features from different collection devices;

[0048] d: Based on the fused feature vectors, the emotion recognition algorithm is used to analyze the user's emotional state, including happiness, sadness, anger, surprise, fear, and disgust;

[0049] e: Based on the identified emotional state, adjust the interaction method of smart products in real time to provide an experience that better matches the user's current emotion.

[0050] Specifically, the feature fusion mechanism uses the following mathematical formula when fusing features from different collection devices:

[0051]

[0052] Where, F i The feature vector extracted from the i-th sensor, α i It is the weight coefficient corresponding to the i-th feature vector, used to adjust the importance of different features during feature fusion.

[0053] Specifically, this includes convolutional neural networks and recurrent neural networks, using long short-term memory networks to capture complex emotional expressions and time dependencies;

[0054] In deep learning algorithms, convolutional layers are used to process the feature vector set F, extracting higher-level features through convolution operations. The mathematical expression for convolution operation is:

[0055]

[0056] Where Y is the output of the convolution operation, F is the input feature map, W is the convolution kernel, (i,j) is the position index on the output feature map, and (m,n) is the size of the convolution kernel;

[0057] After the convolutional layer, the ReLU activation function is applied to increase the nonlinearity of the model. The mathematical expression for ReLU is:

[0058] ReLU(x) = max(0,x)

[0059] Pooling layers are used to reduce the dimensionality of feature maps while preserving important feature information. The mathematical expression for max pooling is:

[0060]

[0061] Where pool_window is the size of the pooling window;

[0062] After multiple convolutional layers, activation functions, and pooling layers, a fully connected layer is used to map the features to the sentiment label space. The output of the fully connected layer is converted into a probability distribution by the Softmax function, representing the confidence level of different sentiment labels. The mathematical expression is as follows:

[0063] O = Softmax(W·Z + b)

[0064] Where O is the output of emotion recognition, W is the weight matrix of the fully connected layer, Z is the output of the last convolutional layer, and b is the bias term;

[0065] Based on the output O of emotion recognition, the interaction mode of smart products is adjusted in real time to provide an experience that is more in line with the user's current emotion.

[0066] Specifically, the training of emotion recognition algorithms includes:

[0067] a: Collect a large amount of sample data with sentiment labels, which comes from public datasets and data collected specifically for this purpose;

[0068] b: By using supervised learning, these data are used to train the emotion recognition algorithm to improve its accuracy in recognizing different emotional states;

[0069] c: Regularly update and optimize the algorithm to adapt to the emotional changes of different user groups and scenarios.

[0070] Specifically, emotion recognition algorithms also consider contextual information, such as the user's historical behavior, current environment, and time, to more accurately identify the user's emotional state.

[0071] As shown above, this method comprehensively collects user physiological data, voice information, facial expressions, and body language by integrating data collection devices such as cameras, microphones, and biometric sensors. Next, key features are extracted through preprocessing steps, and features from different devices are effectively integrated through a feature fusion mechanism. This scheme employs deep learning algorithms, particularly convolutional neural networks, recurrent neural networks, and long short-term memory networks, to capture complex emotional expressions and time dependencies, thereby improving the accuracy of emotion recognition. Supervised learning is used to train the emotion recognition algorithm with a large amount of emotionally labeled sample data, taking into account contextual information such as user history, current environment, and time, further enhancing the algorithm's recognition capabilities. Finally, based on the recognized emotional state, the intelligent product can adjust its interaction methods in real time to provide an experience more aligned with the user's current emotion. The implementation of this method not only improves the user experience but also strengthens the emotional connection between the user and the product, contributing to the continuous development and upgrading of the product.

[0072] Example 2: This example is basically the same as the previous example, except that the interaction method of the smart product is adjusted, including:

[0073] a: Change the colors, brightness, icons, and layout of the display interface to match the user's emotional state;

[0074] b: Adjust the tone, speed, and pitch of voice interaction to better suit the user's emotional needs;

[0075] c: Provide content and suggestions related to your current emotional state, such as recommending music, movies, and articles;

[0076] d: When negative emotions are detected in a user, specific soothing mechanisms are activated, including playing soothing music, providing links to emotional counseling, and offering deep breathing suggestions.

[0077] Specifically, this also includes establishing user emotional profiles to record users' emotional states at different times and in different scenarios, in order to analyze trends in user emotional changes and provide data support for subsequent interaction optimization.

[0078] Specifically, smart products also have self-learning and optimization capabilities, and can adjust emotion recognition algorithms and interaction strategies based on user feedback and real-time data to continuously improve the user experience.

[0079] Specifically, smart products support collaborative work with other smart devices, enabling cross-device emotional interaction experiences by sharing emotion recognition results and interaction strategies.

[0080] As can be seen from the above, this intelligent product can identify the user's emotional state in real time by collecting and analyzing the user's physiological data, voice information, facial expressions and body language, and adjust the user's interaction method accordingly.

[0081] Specific adjustments to the interaction methods include changing the color, brightness, icons, and layout of the display interface; adjusting the tone, speed, and pitch of voice interaction; providing content and suggestions related to the current emotional state; and activating a soothing mechanism when negative emotions are detected. These adjustments not only make the user's interaction experience more in line with their current emotional state, but also add emotional elements to the product, allowing users to feel the humanization and warmth of the product.

[0082] At the same time, the establishment of user emotional profiles and the ability to learn and optimize themselves enable smart products to continuously track and analyze changes in users' emotions, continuously optimize interaction strategies, and improve user experience. This ability allows products to better adapt to the needs of different user groups and scenarios, and improves the flexibility and scalability of products.

[0083] In addition, smart products also support collaborative work with other smart devices. By sharing emotion recognition results and interaction strategies, they can achieve cross-device emotional interaction experiences. This collaborative work can further improve the intelligence level of the product and the user experience, bringing users more convenient and efficient services.

[0084] All standard parts used in this invention can be purchased commercially, and irregularly shaped parts can be customized according to the description and drawings. The specific connection methods for each part all employ conventional methods such as bolts, rivets, and welding, which are mature technologies in the prior art. The machinery, parts, and equipment all use conventional models in the prior art, and the circuit connections also use conventional connection methods in the prior art, which will not be detailed here. Any content not described in detail in this specification belongs to the prior art known to those skilled in the art.

[0085] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. "A plurality of" means two or more, unless otherwise explicitly specified.

[0086] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0087] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first feature is in direct contact with the second feature, or that the first feature is in indirect contact with the second feature through an intermediate medium. Furthermore, "above," "over," and "on top" of the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply that the first feature is at a higher horizontal level than the second feature. "Below," "below," and "under" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply that the first feature is at a lower horizontal level than the second feature.

[0088] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0089] The accompanying drawings of the embodiments disclosed in this invention only involve structures relevant to the embodiments disclosed in this invention. Other structures can be referred to with common designs. Unless otherwise specified, the same embodiment and different embodiments of this invention can be combined with each other.

[0090] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for improving the interactive experience of intelligent products based on emotion recognition, characterized in that, Includes the following steps: a: By collecting devices, including cameras, microphones, and biometric sensors, we continuously collect users' physiological data, voice information, facial expressions, and body language; b: The collected data is preprocessed to remove noise and extract key features to obtain a set of feature vectors; c: Utilize a pre-trained AI-based emotion recognition algorithm and incorporate a feature fusion mechanism to fuse features from different collection devices; d: Based on the fused feature vectors, the emotion recognition algorithm is used to analyze the user's emotional state, including happiness, sadness, anger, surprise, fear, and disgust; e: Based on the identified emotional state, adjust the interaction method of smart products in real time to provide an experience that better matches the user's current emotion.

2. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 1, characterized in that, The feature fusion mechanism uses the following mathematical formula when fusing features from different collection devices: Where, F i The feature vector extracted from the i-th sensor, α i It is the weight coefficient corresponding to the i-th feature vector, used to adjust the importance of different features during feature fusion.

3. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 2, characterized in that, Including convolutional neural networks and recurrent neural networks, using long short-term memory networks to capture complex emotional expressions and time dependencies; In deep learning algorithms, convolutional layers are used to process the feature vector set F, extracting higher-level features through convolution operations. The mathematical expression for convolution operation is: Where Y is the output of the convolution operation, F is the input feature map, W is the convolution kernel, (i,j) is the position index on the output feature map, and (m,n) is the size of the convolution kernel; After the convolutional layer, the ReLU activation function is applied to increase the nonlinearity of the model. The mathematical expression for ReLU is: ReLU(x) = max(0, x) Pooling layers are used to reduce the dimensionality of feature maps while preserving important feature information. The mathematical expression for max pooling is: Where pool_window is the size of the pooling window; After multiple convolutional layers, activation functions, and pooling layers, a fully connected layer is used to map the features to the sentiment label space. The output of the fully connected layer is converted into a probability distribution by the Softmax function, representing the confidence level of different sentiment labels. The mathematical expression is as follows: O = Softmax(W·Z + b) Where O is the output of emotion recognition, W is the weight matrix of the fully connected layer, Z is the output of the last convolutional layer, and b is the bias term; Based on the output O of emotion recognition, the interaction mode of smart products is adjusted in real time to provide an experience that is more in line with the user's current emotion.

4. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 3, characterized in that, The training of emotion recognition algorithms includes: a: Collect a large amount of sample data with sentiment labels, which comes from public datasets and data collected specifically for this purpose; b: By using supervised learning, these data are used to train the emotion recognition algorithm to improve its accuracy in recognizing different emotional states; c: Regularly update and optimize the algorithm to adapt to the emotional changes of different user groups and scenarios.

5. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 4, characterized in that, Emotion recognition algorithms also consider contextual information, such as the user's historical behavior, current environment, and time, to more accurately identify the user's emotional state.

6. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 5, characterized in that, Adjusting the interaction methods of smart products includes: a: Change the colors, brightness, icons, and layout of the display interface to match the user's emotional state; b: Adjust the tone, speed, and pitch of voice interaction to better suit the user's emotional needs; c: Provide content and suggestions related to your current emotional state, such as recommending music, movies, and articles; d: When negative emotions are detected in a user, specific soothing mechanisms are activated, including playing soothing music, providing links to emotional counseling, and offering deep breathing suggestions.

7. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 6, characterized in that, It also includes establishing user emotional profiles to record users' emotional states at different times and in different scenarios, in order to analyze the trends of user emotional changes and provide data support for subsequent interaction optimization.

8. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 1, characterized in that, Smart products also have the ability to learn and optimize themselves, and can adjust emotion recognition algorithms and interaction strategies based on user feedback and real-time data to continuously improve the user experience.

9. The method for improving the interactive experience of intelligent products based on emotion recognition as described in claim 8, characterized in that, Smart products support collaborative work with other smart devices, enabling cross-device emotional interaction experiences by sharing emotion recognition results and interaction strategies.

Images