Emotion change judgment method, device, equipment and medium based on semantic recognition

Abstract

The present invention relates to the technical field of robot emotion recognition. The present invention relates to a method, device, equipment and medium for judging emotion changes based on semantic recognition, including obtaining text input information of a user, performing semantic analysis on the text input information, extracting semantic features, performing emotion classification on the semantic features through an emotion classification model, generating emotion judgment results, obtaining conversation context information corresponding to the user, comprehensively analyzing the conversation context information and the emotion judgment results, obtaining the emotion change trend, and adjusting the robot's behavior and expression in real time according to the emotion change trend. The present application has the effect of improving the ability to recognize and understand the user's emotional state.

Description

Technical Field

[0001] The present invention relates to the technical field of robot emotion recognition, and in particular to an emotion change judgment method, device, equipment and medium based on semantic recognition. Background Art

[0002] With the rapid development of artificial intelligence (AI) technology, speech recognition, natural language processing (NLP) and emotional computing have gradually become the core research areas of intelligent interactive systems. More and more intelligent robots, virtual assistants and other intelligent devices are striving to improve the naturalness of interaction with users and the ability to provide emotional feedback. Existing intelligent interactive systems mainly rely on user language input, such as instructions in the form of text, voice, etc., to perform related operations. However, despite being able to recognize the user's language input content, there are still obvious deficiencies in the ability to recognize and understand the user's emotional state. Summary of the invention

[0003] In order to improve the ability to recognize and understand the user's emotional state, the present application provides an emotion change judgment method, device, equipment and medium based on semantic recognition.

[0004] The above-mentioned invention objective of the present application is achieved through the following technical solutions: A method for judging emotion changes based on semantic recognition, the method for judging emotion changes based on semantic recognition comprising: Acquire text input information from a user, perform semantic analysis on the text input information, and extract semantic features; Performing emotion classification on the semantic features through an emotion classification model to generate an emotion judgment result; Acquire the conversation context information corresponding to the user, and comprehensively analyze the conversation context information and the emotion judgment result to obtain the emotion change trend; According to the emotion change trend, the robot's behavior and expression are adjusted in real time.

[0005] By adopting the above technical solution, the user's text input information is obtained, and the text input information is semantically analyzed and semantic features are extracted, so that key information can be extracted from the user's language expression, helping the system to understand the user's emotional intention more accurately. The emotional features extracted by semantic analysis can better capture the subtle changes in the user's emotions, thereby improving the system's ability to recognize and understand the user's emotional state. The emotional classification model is used to classify the semantic features and generate emotional judgment results, which can convert complex semantic features into specific emotional categories, so that the system can more accurately identify the user's emotional state. Through the classification processing of the emotional classification model, the system can subdivide different emotional states, thereby improving the ability to recognize and understand the user's emotional state. The conversation context information corresponding to the user is obtained, and the conversation context information and the emotional judgment results are comprehensively analyzed to obtain the emotional change trend, which can enable the system to combine context information and historical emotional data to more deeply understand the changes in the user's emotions over time. By analyzing the emotional fluctuations of the user in the conversation, the system can more comprehensively grasp the direction and intensity of emotional changes, thereby significantly improving the ability to recognize and understand the user's emotional state, especially in complex conversation scenarios. According to the trend of emotional changes, the robot's behavior and expression can be adjusted in real time, which enables the robot to respond appropriately to the changes in the user's emotions. This not only helps the robot better understand the user's current emotional state, but also makes its interaction with the user more natural and smooth. By responding to emotional changes in real time, the system can adjust its behavior more sensitively, thereby further improving its ability to recognize and understand the user's emotional state.

[0006] In a preferred example, the present application may be further configured as follows: the obtaining of the user's text input information, and the semantic analysis of the text input information, and the extraction of semantic features, include: Preprocessing the text input information, wherein the preprocessing includes removing stop words, punctuation marks and non-language characters to obtain processed text; A pre-trained language model is used to perform word embedding operations on the processed text to extract semantic features.

[0007] By adopting the above technical solution, the text input information is preprocessed, including removing stop words, punctuation marks and non-language characters, which can remove irrelevant information and simplify the text structure, thereby improving the accuracy of semantic analysis and enhancing the ability to recognize the user's emotional state. The pre-trained language model is used to perform word embedding operations on the processed text to extract semantic features.

[0008] In a preferred example, the present application may be further configured as follows: the emotion classification of the semantic features by the emotion classification model to generate the emotion judgment result includes: Acquire a text data set containing different emotion categories, and preprocess the text data set using natural language processing technology, wherein the preprocessing includes word segmentation, stop word removal, part-of-speech tagging, etc., to obtain text features for training; Constructing a preliminary sentiment classification model, taking the text features as input, and training the preliminary sentiment classification model using a cross entropy loss function to generate a sentiment classification model in training; The parameters of the sentiment classification model in training are optimized by a back propagation algorithm until the sentiment classification model in training reaches a preset convergence condition, the training is completed, and the sentiment classification model is obtained.

[0009] By adopting the above technical solution, a text data set containing different emotion categories is obtained, and the text data set is preprocessed using natural language processing technology, which can ensure that the input data of the trained model has good quality and feature representation, thereby improving the accuracy of the emotion classification model in recognizing user emotions. Constructing a preliminary emotion classification model and training it using the cross entropy loss function can effectively distinguish different emotion categories, improve the classification effect of the model, and thus enhance the ability to understand the user's emotional state. By optimizing the model parameters through the back propagation algorithm and achieving the preset convergence conditions, it can ensure that the emotion classification model achieves the best performance, thereby more accurately capturing the changing trend of user emotions.

[0010] In a preferred example, the present application may be further configured as follows: the emotion classification model is used to analyze the semantic features for emotion categories, and the emotion judgment result is generated, including: Inputting the semantic features into the sentiment classification model, classifying the semantic features into sentiment labels using a classification algorithm, and obtaining probability scores of multiple sentiment categories; The probability scores are normalized using a Softmax function to obtain a probability value for each emotion category; The probability values ​​are compared, and the emotion category corresponding to the maximum probability value is selected as the emotion judgment result.

[0011] By adopting the above technical solution, inputting semantic features into the sentiment classification model and using the classification algorithm to classify sentiment labels, the user's emotional state can be preliminarily identified, ensuring that the classification of different sentiment categories is more accurate, thereby improving the ability to understand the user's emotions. Using the Softmax function to normalize the probability scores can standardize the probability distribution of multiple sentiment categories, making the model's output more interpretable, thereby enhancing the accuracy of emotion recognition. Comparing the probability values ​​and selecting the sentiment category corresponding to the maximum probability value as the sentiment judgment result can effectively identify the most likely user emotional state, thereby improving the accuracy of judging the user's current emotions.

[0012] In a preferred example, the present application may be further configured as follows: the acquiring of the conversation context information corresponding to the user, and the comprehensive analysis of the conversation context information and the emotion judgment result to obtain the emotion change trend include: The context information includes the previous and next conversation content, conversation scene, semantic information and user historical conversation data. Based on the context information and emotion judgment results, the user's emotion change characteristics in the continuous conversation are analyzed to determine the direction and amplitude of the emotion fluctuation; The emotion change trend of the user is determined according to the direction and amplitude of the emotion fluctuation.

[0013] By adopting the above technical solution, the context information includes the previous and subsequent conversation content, conversation scene, semantic information, and user historical conversation data. Based on the context information and emotion judgment results, the characteristics of the user's emotional changes in continuous conversations are analyzed, which can deeply understand the user's emotional change pattern. It is not just based on the emotion analysis of a single moment, but integrates historical conversations and scene information, thereby improving the accuracy of identifying user emotional fluctuations. According to the direction and amplitude of emotional fluctuations, the user's emotional change trend is determined, which can more accurately capture the direction of the user's emotional changes, help the system to adjust the dialogue strategy and robot response in time, and thus enhance the understanding and response capabilities of user emotional changes.

[0014] In a preferred example, the present application can be further configured as follows: adjusting the robot's behavior and expression in real time according to the emotion change trend includes: According to the emotion change trend, judging the current emotional state of the user, the current emotional state includes positive emotion, negative emotion and neutral emotion; Selecting an expression adjustment scheme that matches the current emotional state of the user from a preset behavior and expression adjustment strategy library, wherein the expression adjustment scheme includes a variety of facial expressions, intonations, and body movements; Based on the expression adjustment scheme, the behavior and expression of the robot are adjusted in real time.

[0015] By adopting the above technical solution, the user's current emotional state, including positive emotions, negative emotions and neutral emotions, can be judged according to the trend of emotional changes, and the user's emotional state can be accurately captured, and the user's emotional category can be identified in real time, thereby improving the system's sensitivity and responsiveness to the user's emotions. The expression adjustment scheme that matches the user's current emotional state is selected from the preset behavior and expression adjustment strategy library. The expression adjustment scheme includes a variety of facial expressions, intonations and body movements, which can dynamically adjust the robot's behavior according to the user's emotional state, making the interaction more natural and personalized, thereby improving the user experience satisfaction. Based on the expression adjustment scheme, the robot's behavior and expression are adjusted in real time to ensure that the robot responds quickly to the user's emotions. The expression, intonation, and body movements are adjusted to match the user's current emotional state, thereby improving the emotional adaptability and intelligence level of human-computer interaction.

[0016] In a preferred example, the present application may be further configured as follows: selecting an expression adjustment scheme matching the current emotional state from a preset behavior and expression adjustment strategy library, wherein the expression adjustment scheme includes facial expressions, intonations and body movements, and further comprising: Obtain historical emotional information and feedback information when users interact with robots, analyze users' behavioral responses and satisfaction in multiple emotional states, and generate feedback results; Based on the feedback results, the behavior and expression adjustment strategies are optimized and added to obtain updated behavior and expression adjustment strategies, and the updated behavior and expression adjustment strategies are stored in the preset behavior and expression adjustment strategy library.

[0017] By adopting the above technical solution, the historical emotional information and feedback information of the user when interacting with the robot are obtained, the user's behavioral response and satisfaction in multiple emotional states are analyzed, and feedback results are generated, which can deeply understand the needs and preferences of users in different emotional states, thereby providing data support for subsequent interactions to improve the robot's emotional recognition and response capabilities. Based on the feedback results, the behavior and expression adjustment strategies are optimized and added, and the updated behavior and expression adjustment strategies are obtained, which can enable the robot to continuously improve its performance during the interaction with the user to adapt to the user's changing emotional needs, thereby improving the flexibility and intelligence level of the robot. The updated behavior and expression adjustment strategies are stored in the preset behavior and expression adjustment strategy library, which can realize the dynamic update and iteration of the strategy, ensuring that the robot can continuously reflect the changes in user emotions in actual applications, thereby enhancing the user's interactive experience and satisfaction.

[0018] The second object of the invention is achieved by the following technical solutions: An emotion change judgment device based on semantic recognition, the emotion change judgment device based on semantic recognition comprising: A text semantic analysis module is used to obtain text input information from a user, perform semantic analysis on the text input information, and extract semantic features; The sentiment classification module is used to classify the semantic features into sentiments through a sentiment classification model to generate sentiment judgment results; A context comprehensive analysis module is used to obtain the user's conversation context information, and comprehensively analyze the conversation context information and the emotion judgment result to obtain the emotion change trend; The behavior and expression adjustment module is used to adjust the robot's behavior and expression in real time according to the emotion change trend.

[0019] By adopting the above technical solution, The third objective of the present application is achieved through the following technical solutions: A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned method for determining emotion changes based on semantic recognition when executing the computer program.

[0020] The fourth objective of the present application is achieved through the following technical solutions: A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for determining emotion changes based on semantic recognition.

[0021] In summary, the present application includes at least one of the following beneficial technical effects: 1. Obtain the user's text input information, perform semantic analysis on the text input information, and extract semantic features. This can extract key information from the user's language expression and help the system understand the user's emotional intention more accurately. The emotional features extracted through semantic analysis can better capture the subtle changes in the user's emotions, thereby improving the system's ability to recognize and understand the user's emotional state. The emotional classification model is used to classify the semantic features and generate emotional judgment results. It can convert complex semantic features into specific emotional categories, enabling the system to more accurately identify the user's emotional state. Through the classification processing of the emotional classification model, the system can subdivide different emotional states, thereby improving the ability to recognize and understand the user's emotional state. Obtain the conversation context information corresponding to the user, conduct a comprehensive analysis of the conversation context information and the emotional judgment results, and obtain the emotional change trend. This enables the system to combine context information and historical emotional data to more deeply understand the changes in user emotions over time. By analyzing the emotional fluctuations of the user in the conversation, the system can more comprehensively grasp the direction and intensity of emotional changes, thereby significantly improving the ability to recognize and understand the user's emotional state, especially in complex conversation scenarios. According to the trend of emotional changes, the robot's behavior and expression can be adjusted in real time, which enables the robot to respond appropriately to the changes in the user's emotions. This not only helps the robot better understand the user's current emotional state, but also makes its interaction with the user more natural and smooth. By responding to emotional changes in real time, the system can adjust its behavior more sensitively, thereby further improving its ability to recognize and understand the user's emotional state; 2. Context information includes the previous and subsequent conversation content, conversation scenes, semantic information, and user historical conversation data. Based on context information and emotion judgment results, the characteristics of user emotion changes in continuous conversations are analyzed, which can deeply understand the user's emotion change pattern. It is not just based on the emotion analysis of a single moment, but integrates historical conversations and scene information, thereby improving the accuracy of identifying user emotion fluctuations. According to the direction and amplitude of emotion fluctuations, the user's emotion change trend is determined, which can more accurately capture the direction of user emotion changes, help the system to adjust the conversation strategy and robot response in time, and thus enhance the understanding and response capabilities of user emotion changes; 3. Obtain historical emotional information and feedback information when users interact with robots, analyze users' behavioral responses and satisfaction in multiple emotional states, and generate feedback results, which can provide in-depth understanding of users' needs and preferences in different emotional states, thereby providing data support for subsequent interactions to improve the robot's emotional recognition and response capabilities. Based on the feedback results, optimize and add behavior and expression adjustment strategies, and obtain updated behavior and expression adjustment strategies, which can enable the robot to continuously improve its performance during the interaction with users to adapt to the user's changing emotional needs, thereby improving the robot's flexibility and intelligence level. Storing the updated behavior and expression adjustment strategy in the preset behavior and expression adjustment strategy library can realize dynamic update and iteration of the strategy, ensuring that the robot can continuously reflect changes in user emotions in actual applications, thereby enhancing the user's interactive experience and satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a method for determining emotion changes based on semantic recognition in an embodiment of the present application; Figure 2 It is a flowchart for implementing step S10 in the emotion change judgment method based on semantic recognition in one embodiment of the present application; Figure 3 It is a flowchart of the implementation before step S20 in the emotion change judgment method based on semantic recognition in one embodiment of the present application; Figure 4 It is a flowchart for implementing step S20 in the emotion change judgment method based on semantic recognition in one embodiment of the present application; Figure 5 It is a flowchart for implementing step S30 in the emotion change judgment method based on semantic recognition in an embodiment of the present application; Figure 6 It is a flowchart for implementing step S40 in the emotion change judgment method based on semantic recognition in an embodiment of the present application; Figure 7 It is a flowchart for implementing step S402 in the emotion change judgment method based on semantic recognition in an embodiment of the present application; Figure 8 It is a principle block diagram of an emotion change judgment device based on semantic recognition in one embodiment of the present application; Fig. 9 It is a schematic diagram of a device in an embodiment of the present application. DETAILED DESCRIPTION

[0023] The present application is further described in detail below in conjunction with the accompanying drawings.

[0024] In one embodiment, if Figure 1As shown, the present application discloses a method for judging emotion changes based on semantic recognition, which specifically includes the following steps: S10: Obtain text input information from the user, perform semantic analysis on the text input information, and extract semantic features.

[0025] Specifically, the robot receives the user's voice signal through a microphone or other input device, and converts it into text form through voice recognition technology. In order to improve the accuracy of text analysis, the system first preprocesses the text, including removing stop words, punctuation marks, and non-language characters to ensure that the core information in the text can be accurately extracted. Next, the system performs word segmentation on the text, uses techniques such as word form restoration to restore the inflected words to their original form, and understands the sentence structure and semantic relationships in the text through dependency syntax analysis. Next, the system uses a pre-trained natural language processing model to perform semantic analysis on the preprocessed text and extract high-dimensional semantic features that represent the user's speech content.

[0026] Further, the system refers to a system for monitoring and analyzing the emotional state of the user in real time, and dynamically adjusting the robot's behavior and expression based on these emotional states.

[0027] S20: Use the sentiment classification model to classify the semantic features and generate sentiment judgment results. Specifically, after extracting semantic features, the system inputs these features into a pre-trained sentiment classification model that can identify the differences between different emotion categories (such as anger, joy, sadness, and surprise). The sentiment classification model processes semantic features through a multi-layer neural network structure, first passing through a series of fully connected layers or convolutional layers to extract more abstract emotional information. The model uses the Softmax layer to calculate the probability of each emotion category and normalize it so that the sum of the probability values ​​of different emotion categories is 1. By comparing the probability values, the system determines the emotion category with the highest probability as the final emotion judgment result. For example, if the emotion category probability of "anger" is the highest, the system will determine that the user is currently in an angry state.

[0028] S30: Acquire the conversation context information corresponding to the user, and comprehensively analyze the conversation context information and the emotion judgment result to obtain the emotion change trend.

[0029] Specifically, the system not only processes the current semantic and emotional judgment results, but also integrates the user's conversation context information for a more comprehensive sentiment analysis. Context information includes the content before and after the current conversation, the conversation scene, the user's historical conversation data, and the long-term interaction record between the user and the robot. By performing time series analysis on this context information, the system can understand the user's emotional fluctuations in multiple rounds of conversation. Specifically, the system analyzes the emotional judgment results of the current conversation and compares them with the emotional states of the previous rounds of conversation to determine whether the user's emotions tend to be positive or negative, and the magnitude of the emotional changes.

[0030] S40: Adjust the robot’s behavior and expression in real time according to the trend of emotional changes.

[0031] Specifically, when the system determines the changing trend of the user's emotions, it will select appropriate robot behavior and expression adjustment strategies based on the current emotional state. If the user's emotions tend to be positive, for example, the user's emotions change from a neutral state to a happy state, the robot will enhance the interaction effect by showing a smile, a gentle tone, and open body movements. Conversely, if the user's emotions turn to negative, such as anger or sadness, the robot will appropriately adjust to a more comforting tone, and the expression will become more caring or cautious, or even reduce body movements to avoid angering the user. When the system adjusts the robot's expression in real time, it will also select an appropriate response strategy based on the preset strategy library, which contains a variety of predefined expression combinations and intonation patterns. These strategies are continuously optimized through previous user feedback. In addition, the system will monitor the user's reaction in real time and continue to adjust the robot's behavior according to the changes in the user's expression and tone to ensure that the robot's response always meets the user's emotional needs. For example, when the user's emotions gradually ease, the robot will gradually return to a relaxed expression and tone, making the interaction process more natural and smooth.

[0032] In one embodiment, if Figure 2 As shown, in step S10, the user's text input information is obtained, and semantic analysis is performed on the text input information to extract semantic features, including: S101: Preprocessing the text input information, where the preprocessing includes removing stop words, punctuation marks and non-language characters to obtain processed text.

[0033] Specifically, the system standardizes the text information input by the user, including converting all characters to lowercase to ensure case-insensitivity when processing the text. Then, the system removes stop words from the text through regular expressions or natural language processing tools (such as NLTK or SpaCy). The stop word list usually contains common function words (such as "de", "shi", "zai", etc.), which contribute less to semantic analysis. In addition, the system also removes punctuation marks and non-verbal characters, such as commas, periods, question marks, and non-verbal symbols, to simplify the text content. During the processing, the system retains keyword and entity information to ensure the integrity of semantic information without loss. Finally, after these preprocessing operations, the text is converted into a form that only contains meaningful words and phrases.

[0034] S102: Perform word embedding operations on the processed text using a pre-trained language model to extract semantic features.

[0035] In this embodiment, the pre-trained language model refers to the BERT model. BERT is a context-sensitive language model that can capture the semantic changes of words in different contexts.

[0036] Specifically, the system inputs the processed text sequence into the BERT model. The model encodes each word in the text through the self-attention mechanism to generate dynamic word embedding vectors. These embedding vectors not only represent the semantics of individual words but also consider the position and context information of the word in the sentence. Therefore, the same word can generate different embedding vectors in different sentences. In this way, BERT can better handle polysemous words and complex semantic relationships. After generating the word embeddings, the system uses these vectors as semantic features.

[0037] In one embodiment, as Figure 3 shown, before step S20, that is, before generating an emotion judgment result by classifying the semantic features through an emotion classification model, it includes: S11: Obtain a text data set containing different emotion categories, and use natural language processing techniques to preprocess the text data set. The preprocessing includes word segmentation, stop word removal, and part-of-speech tagging, etc., to obtain text features for training.

[0038] Specifically, the system first collects a text dataset containing various emotion tags from multiple sources, such as social media comments, customer service conversation records, and user feedback. Then, natural language processing techniques are used to preprocess the collected text data. The specific preprocessing process includes tokenizing each text, breaking long texts into individual words or phrases for subsequent analysis; removing stop words, that is, filtering out common words that have no substantial meaning for emotion judgment (such as "de", "le", "he", etc.); in addition, part-of-speech tagging is also performed to label the grammatical role of each word for subsequent use in feature extraction. After these processes, structured text features are finally obtained, and these features will be used as the input of the sentiment classification model to ensure that the model can accurately learn the manifestation forms of emotions.

[0039] S12: Construct a preliminary sentiment classification model, use the text features as input, and train the preliminary sentiment classification model using the cross-entropy loss function to generate the sentiment classification model during training.

[0040] Specifically, taking the text features obtained through preprocessing as the input of the model, the system first performs mapping of feature vectors, and then learns these features through a multi-layer neural network. During the training process, the cross-entropy loss function is used to evaluate the difference between the predicted output of the model and the actual emotion labels. The cross-entropy loss function can effectively quantify the accuracy of the classification results. According to this loss value, the system will gradually update the parameters of the model, so that the performance of the model on the training set is continuously improved, thereby generating a sentiment classification model during training.

[0041] S13: Optimize the parameters of the sentiment classification model during training through the backpropagation algorithm until the sentiment classification model during training reaches the preset convergence condition, complete the training, and obtain the sentiment classification model.

[0042] The backpropagation algorithm refers to calculating the gradient of the loss function with respect to the model parameters to guide the model to update in the direction of reducing the prediction error.

[0043] Specifically, the system inputs the training data into the model in batches, calculates the loss of each batch, and adjusts the parameters of the model through the gradient descent method. This process will continue until the performance of the model on the validation set reaches the preset convergence condition, that is, when the change in the loss function is less than the set threshold or after a certain number of iterations, the system considers that the model has reached the best state. After the training is completed, the finally generated sentiment classification model will be able to accurately classify the sentiment of the input text.

[0044] In one embodiment, as Figure 4 shown, in step S20, that is, through the sentiment classification model, the semantic features are analyzed for emotion categories to generate emotion judgment results, including: S201: Input the semantic features into the sentiment classification model, classify the semantic features into multiple categories of sentiment labels, and calculate the probability score of each sentiment category.

[0045] Specifically, after the semantic features are input into the sentiment classification model, the model first analyzes the semantic features of the text through a pre-trained deep learning algorithm. The classification algorithm will classify different sentiment categories (such as positive, negative, neutral, etc.) based on the training experience of the sentiment classification model. The model generates a probability score for each sentiment category by performing deep learning on the context of the input semantic features and sentiment-related words.

[0046] S202: Use the Softmax function to normalize the probability scores to obtain the probability value of each emotion category.

[0047] Specifically, the system uses the Softmax function to normalize the probability scores generated in the previous step. The Softmax function converts each score into a probability value between 0 and 1, and the sum of these probability values ​​is 1. This means that the probability value of each emotion category represents the relative possibility that the input text is classified as that emotion.

[0048] Furthermore, the Softmax function refers to an activation function used for multi-classification problems, which converts the original output of multiple classifications into a probability distribution. The role of the Softmax function is to normalize the input sentiment category scores so that the score of each sentiment category falls between 0 and 1, and the sum of the probabilities of all categories is 1.

[0049] S203: Compare the probability values ​​of each emotion category, and select the emotion category with the highest probability value as the emotion judgment result.

[0050] Specifically, the system compares the probability values ​​of each emotion category obtained through Softmax processing to determine which emotion category has the highest probability value. The system will identify the emotion category with the highest probability value and use it as the final emotion judgment result. This result not only reflects the subjective emotion expressed by the user's text input, but can also be further used to adjust the system's behavior and expression to better respond to the user's emotional state.

[0051] In one embodiment, if Figure 5 As shown, in step S30, the conversation context information corresponding to the user is obtained, and the conversation context information and the emotion judgment result are comprehensively analyzed to obtain the emotion change trend, including: S301: Context information includes previous and subsequent conversation content, conversation scenes, semantic information, and user historical conversation data. Based on the context information and emotion judgment results, the user's emotion change characteristics in continuous conversations are analyzed to determine the direction and amplitude of emotion fluctuations.

[0052] Specifically, the system first obtains and integrates the contextual information of the user's current conversation, including the semantic structure of the previous and subsequent conversation content, the scene of the current conversation (such as interviews, small talk, complaints, etc.), and the user's historical conversation records. By analyzing this information, the system can identify the development trajectory of the user's emotions throughout the conversation. Combined with the results of emotional judgment, the system will focus on the continuous change characteristics of the user's emotions, such as the gradual change from calm to anxiety, from happiness to sadness, etc. By analyzing the emotional fluctuations in continuous conversations, the system can calculate the direction of emotional changes (such as positive, negative or stable) and the amplitude (such as slight changes or drastic fluctuations), thereby obtaining a dynamic understanding of the user's current emotional state.

[0053] Determine the user's emotional change trend based on the direction and amplitude of emotional fluctuations.

[0054] Specifically, based on the direction and amplitude of the emotion fluctuations obtained from the previous analysis, the system can further infer the user's emotion change trend. For example, if the user's emotion fluctuations are small and tend to be stable in multiple rounds of dialogue, the system may believe that the user's emotions remain stable; if the direction of the emotion fluctuations is significant and accompanied by a large amplitude change, it can be judged that the user's emotions are in a clear upward or downward trend. The system will combine the current dialogue scene with historical emotion data to conduct a comprehensive assessment of the emotion change trend and generate a more accurate prediction of the emotion trend.

[0055] In one embodiment, if Figure 6 As shown, in step S40, the robot's behavior and expression are adjusted in real time according to the trend of emotional changes, including: S401: judging the current emotional state of the user according to the emotion change trend, where the current emotional state includes positive emotion, negative emotion and neutral emotion.

[0056] Specifically, the system first makes a real-time judgment on the user's emotional state based on the previously acquired emotional change trend and the contextual information of the current conversation. Emotional states can be classified into three main types: positive emotions (such as happiness and excitement), negative emotions (such as sadness and anger), and neutral emotions (such as calmness and calmness). By continuously monitoring and analyzing the user's emotions, the system can not only judge the user's instantaneous emotions, but also identify the user's emotional change trends, such as the emotion gradually turning negative or gradually recovering from a negative state. Based on this dynamic analysis, the system ensures that it can make accurate judgments on the user's emotions at each stage of the conversation.

[0057] S402: Selecting an expression adjustment scheme that matches the user's current emotional state from a preset behavior and expression adjustment strategy library, where the expression adjustment scheme includes a variety of facial expressions, intonations, and body movements.

[0058] Specifically, after determining the user's emotional state, the system will select the most suitable expression and behavior for the current emotional state from a predefined expression adjustment strategy library. The strategy library contains a variety of adjustment schemes corresponding to different emotional states, and each scheme includes multiple dimensions of behavioral adjustments, such as the robot's facial expressions (smile, frown, surprise, etc.), tone changes (soft, firm, relaxed, etc.), and body movements (such as leaning the body, gesture assistance, etc.). For example, when the user shows positive emotions, the system will choose a dynamic and interactive behavior scheme, and the robot may respond to the user through a cheerful tone, a smiling expression, and open body movements; when the user is in a negative mood, the system may choose a soothing adjustment scheme, and the expression and movement will become softer to avoid further stimulating the user's negative emotions.

[0059] S403: Based on the expression adjustment scheme, adjust the robot's behavior and expression in real time.

[0060] Specifically, once a suitable expression adjustment scheme is selected, the system will instruct the robot to make corresponding expression and behavior adjustments immediately. The real-time adjustment process includes multiple steps. First, the system will control the robot's facial expressions (such as facial muscle simulations such as smiling and frowning), voice tone (such as increasing or decreasing speech speed, changing volume or pitch), and its body movements (such as arm movement, body posture changes, etc.) according to the selected scheme. This process ensures that the robot can show behaviors that match the user's current emotional state when interacting with the user. For example, if the user's emotional state changes from negative to neutral or positive, the robot will switch to a more positive interaction mode in time, with facial expressions becoming more friendly and pleasant, and the tone of voice becoming lighter; on the contrary, if the user's emotions become more negative, the robot will show empathy, adjust the expression to be soothing, and slow down the tone of voice.

[0061] In one embodiment, if Figure 6 As shown, in step S402, an expression adjustment scheme matching the current emotional state is selected from a preset behavior and expression adjustment strategy library, and the expression adjustment scheme includes facial expressions, intonation and body movements, and also includes: S4021: Obtain historical emotional information and feedback information when the user interacts with the robot, analyze the user's behavioral responses and satisfaction under multiple emotional states, and generate feedback results.

[0062] Specifically, the system will record and store the user's emotional state, behavioral response, and related feedback information during multiple interactions between the user and the robot. This information not only includes the user's immediate emotional expression in a specific emotional state, but also captures the user's response to the robot's behavior and expression adjustments in different situations. For example, the system will record the user's facial expression changes, voice emotions, and interactive behaviors (such as whether they smile, whether they show anxiety, etc.) during the interaction, and evaluate whether the robot's adjustments are appropriate and effective through the user's actual feedback (such as satisfaction ratings, evaluations after interaction, etc.). Through long-term data accumulation, the system can better understand the user's preferences and needs in different emotional states, thereby generating a comprehensive feedback result that includes user behavioral responses and satisfaction. This feedback result will be used to further optimize the emotional interaction experience between the robot and the user, so that the robot can better meet the user's emotional needs.

[0063] S4022: Based on the feedback results, optimize and add behavior and expression adjustment strategies to obtain updated behavior and expression adjustment strategies, and store the updated behavior and expression adjustment strategies in a preset behavior and expression adjustment strategy library.

[0064] Specifically, the system optimizes existing behavior and expression adjustment strategies based on user feedback. First, by analyzing the feedback results, the system will find out which behavior and expression adjustments can effectively improve user satisfaction, and which adjustment plans do not meet user expectations or do not produce positive effects under certain emotional states. Based on these data, the system can improve existing adjustment strategies, such as optimizing facial expression details for different user groups or specific emotional states, adjusting the range of voice changes, and adding more diverse body movements. In addition, the system will automatically generate new adjustment strategies based on user feedback and historical emotional information, and expand the behavior and expression plans in the strategy library. Finally, the system stores the updated strategies in the preset behavior and expression adjustment strategy library to ensure that the latest and best adjustment plans can be used in future interactions with users.

[0065] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0066] In one embodiment, a device for judging emotion changes based on semantic recognition is provided, and the device for judging emotion changes based on semantic recognition corresponds one-to-one to the method for judging emotion changes based on semantic recognition in the above embodiment. Figure 8 As shown, the emotion change judgment device based on semantic recognition includes a text semantic analysis module, an emotion classification module, a context comprehensive analysis module and a behavior expression adjustment module. The detailed description of each functional module is as follows: The text semantic analysis module is used to obtain the user's text input information, perform semantic analysis on the text input information, and extract semantic features; The sentiment classification module is used to classify the sentiment of semantic features through the sentiment classification model and generate sentiment judgment results; The context comprehensive analysis module is used to obtain the user's conversation context information, conduct a comprehensive analysis of the conversation context information and the emotion judgment results, and obtain the emotion change trend; The behavior and expression adjustment module is used to adjust the robot's behavior and expression in real time according to the trend of emotional changes.

[0067] Optionally, the text semantic analysis module includes: The text processing submodule is used to pre-process the text input information, including removing stop words, punctuation marks and non-language characters to obtain processed text; The semantic feature submodule is used to perform word embedding operations on the processed text using a pre-trained language model to extract semantic features.

[0068] Optionally, the sentiment classification module includes: The text dataset acquisition module is used to acquire text datasets containing different emotion categories and preprocess the text datasets using natural language processing technology, including word segmentation, stop word removal and part-of-speech tagging, to obtain text features for training; The model building module is used to build a preliminary sentiment classification model, take text features as input, train the preliminary sentiment classification model using the cross entropy loss function, and generate a sentiment classification model in training; The model training completion module is used to optimize the parameters of the sentiment classification model in training through the back propagation algorithm until the sentiment classification model in training reaches the preset convergence condition, completes the training, and obtains the sentiment classification model.

[0069] Optionally, the sentiment classification module includes: The probability score submodule is used to input the semantic features into the sentiment classification model, classify the semantic features into multiple sentiment labels, and calculate the probability score of each sentiment category; The probability value submodule is used to normalize the probability scores using the Softmax function to obtain the probability value of each emotion category; The emotion judgment result submodule is used to compare the probability values ​​of each emotion category and select the emotion category with the highest probability value as the emotion judgment result.

[0070] Optional contextual analysis modules include: The emotion fluctuation and amplitude submodule is used for context information including previous and subsequent conversation content, conversation scenes, semantic information, and user historical conversation data. Based on the context information and emotion judgment results, it analyzes the characteristics of the user's emotion changes in continuous conversations and determines the direction and amplitude of emotion fluctuations. The emotion change trend submodule is used to determine the user's emotion change trend based on the direction and amplitude of emotion fluctuations.

[0071] Optionally, the behavior expression adjustment module includes: The emotional state submodule is used to judge the user's current emotional state based on the trend of emotional changes. The current emotional state includes positive emotion, negative emotion and neutral emotion; The expression adjustment scheme submodule is used to select an expression adjustment scheme that matches the user's current emotional state from a preset behavior and expression adjustment strategy library. The expression adjustment scheme includes a variety of facial expressions, intonations, and body movements; The robot adjustment submodule is used to adjust the robot's behavior and expression in real time based on the expression adjustment scheme.

[0072] Optionally, the expression adjustment scheme submodule includes: Feedback result unit, used to obtain historical emotional information and feedback information when the user interacts with the robot, analyze the user's behavioral response and satisfaction under multiple emotional states, and generate feedback results; The expression adjustment strategy library updating unit is used to optimize and add behavior and expression adjustment strategies based on feedback results, obtain updated behavior and expression adjustment strategies, and store the updated behavior and expression adjustment strategies in a preset behavior and expression adjustment strategy library.

[0073] For the specific limitations of the emotion change judgment device based on semantic recognition, please refer to the limitations of the emotion change judgment method based on semantic recognition above, which will not be repeated here. Each module in the above-mentioned emotion change judgment device based on semantic recognition can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0074] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used for a preset behavior and expression adjustment strategy library. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for judging emotion changes based on semantic recognition is implemented.

[0075] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the following steps when executing the computer program: Obtain the user's text input information, perform semantic analysis on the text input information, and extract semantic features; The sentiment classification model is used to classify the semantic features and generate sentiment judgment results; Obtain the conversation context information corresponding to the user, conduct a comprehensive analysis of the conversation context information and the emotion judgment results, and obtain the emotion change trend; Adjust the robot's behavior and expression in real time based on emotional trends.

[0076] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: Obtain the user's text input information, perform semantic analysis on the text input information, and extract semantic features; The sentiment classification model is used to classify the semantic features and generate sentiment judgment results; Obtain the conversation context information corresponding to the user, conduct a comprehensive analysis of the conversation context information and the emotion judgment results, and obtain the emotion change trend; Adjust the robot's behavior and expression in real time based on emotional trends.

[0077] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0078] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0079] The embodiments described above are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for judging emotion changes based on semantic recognition, characterized in that: The emotion change judgment method based on semantic recognition includes: Acquire text input information from a user, perform semantic analysis on the text input information, and extract semantic features; Performing emotion classification on the semantic features through an emotion classification model to generate an emotion judgment result; Acquire the conversation context information corresponding to the user, and comprehensively analyze the conversation context information and the emotion judgment result to obtain the emotion change trend; According to the emotion change trend, the robot's behavior and expression are adjusted in real time.

2. The method for determining emotion changes based on semantic recognition according to claim 1, characterized in that: The obtaining of the user's text input information, and performing semantic analysis on the text input information to extract semantic features includes: Preprocessing the text input information, wherein the preprocessing includes removing stop words, punctuation marks and non-language characters to obtain processed text; A pre-trained language model is used to perform word embedding operations on the processed text to extract semantic features.

3. The emotion change judgment method based on semantic recognition according to claim 1 is characterized in that: The emotion classification of the semantic features by the emotion classification model to generate the emotion judgment result includes: Acquire a text data set containing different emotion categories, and preprocess the text data set using natural language processing technology, wherein the preprocessing includes word segmentation, stop word removal, part-of-speech tagging, etc., to obtain text features for training; Constructing a preliminary sentiment classification model, taking the text features as input, and training the preliminary sentiment classification model using a cross entropy loss function to generate a sentiment classification model in training; The parameters of the sentiment classification model in training are optimized by a back propagation algorithm until the sentiment classification model in training reaches a preset convergence condition, the training is completed, and the sentiment classification model is obtained.

4. The method for determining emotion changes based on semantic recognition according to claim 1, characterized in that: The emotion classification model is used to analyze the semantic features for emotion categories to generate emotion judgment results, including: Inputting the semantic features into the sentiment classification model, classifying the semantic features into multiple categories of sentiment labels, and calculating the probability score of each sentiment category; Using the Softmax function to normalize the probability scores of the emotion categories to obtain the probability value of each emotion category; The probability values ​​of each emotion category are compared, and the emotion category with the highest probability value is selected as the emotion judgment result.

5. The emotion change judgment method based on semantic recognition according to claim 1, characterized in that: The acquiring the conversation context information corresponding to the user, and performing a comprehensive analysis on the conversation context information and the emotion judgment result to obtain the emotion change trend, includes: The context information includes the previous and next conversation content, conversation scene, semantic information and user historical conversation data. Based on the context information and emotion judgment results, the user's emotion change characteristics in the continuous conversation are analyzed to determine the direction and amplitude of the emotion fluctuation; The emotion change trend of the user is determined according to the direction and amplitude of the emotion fluctuation.

6. The method for determining emotion changes based on semantic recognition according to claim 1, characterized in that: The real-time adjustment of the robot's behavior and expression according to the emotion change trend includes: According to the emotion change trend, judging the current emotional state of the user, the current emotional state includes positive emotion, negative emotion and neutral emotion; Selecting an expression adjustment scheme that matches the current emotional state of the user from a preset behavior and expression adjustment strategy library, wherein the expression adjustment scheme includes a variety of facial expressions, intonations, and body movements; Based on the expression adjustment scheme, the behavior and expression of the robot are adjusted in real time.

7. The method for determining emotion changes based on semantic recognition according to claim 6, characterized in that: The expression adjustment scheme matching the current emotional state is selected from a preset behavior and expression adjustment strategy library, wherein the expression adjustment scheme includes facial expressions, intonation and body movements, and further includes: Obtain historical emotional information and feedback information when users interact with robots, analyze users' behavioral responses and satisfaction in multiple emotional states, and generate feedback results; Based on the feedback results, the behavior and expression adjustment strategies are optimized and added to obtain updated behavior and expression adjustment strategies, and the updated behavior and expression adjustment strategies are stored in the preset behavior and expression adjustment strategy library.

8. An emotion change judgment device based on semantic recognition, characterized in that: The emotion change judgment device based on semantic recognition includes: A text semantic analysis module is used to obtain text input information from a user, perform semantic analysis on the text input information, and extract semantic features; The sentiment classification module is used to classify the semantic features into sentiments through a sentiment classification model to generate sentiment judgment results; A context comprehensive analysis module is used to obtain the user's conversation context information, and comprehensively analyze the conversation context information and the emotion judgment result to obtain the emotion change trend; The behavior and expression adjustment module is used to adjust the robot's behavior and expression in real time according to the emotion change trend.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the emotion change judgment method based on semantic recognition as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the emotion change judgment method based on semantic recognition as described in any one of claims 1 to 7 are implemented.

Images