Emotional dialogue method of terminal digital person based on large model

Abstract

The invention relates to the technical field of artificial intelligence, in particular to an emotional dialogue method of a terminal digital person based on a large model, which comprises the following steps: after a user sends a query, a system retrieves the most matched memory fragment by combining the semantic and emotion information of the query, so that the dialogue content is consistent with both semantic logic and emotional information when being generated; and resonance with the emotion of the user can be generated. According to the method, the generated dialogue can be more humanized through an emotion perception retrieval and generation mechanism, the emotional state of the user can be effectively adapted, and the emotional interaction effect of a terminal digital person is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to an emotional dialogue method for a terminal digital human based on a large model. Background Art

[0002] In recent years, terminal digital human technology based on large-scale language models (LLM) has been widely used in role-playing, customer service, and virtual assistants. By using Retrieval-Augmented Generation (RAG) technology, digital humans are able to retrieve information from the knowledge base to generate responses that better meet user needs. However, existing RAG technologies mostly rely on semantic similarity, but ignore the emotional dimension, resulting in insufficient emotional resonance in conversations. Emotional factors are not adequately considered in digital human dialogue systems, and the generated dialogue content often lacks emotional consistency, making it difficult to effectively convey the emotional characteristics and empathy of the character. This defect causes terminal digital humans to have significant disadvantages in user experience and emotional identification, affecting the interactivity and immersion of the conversation. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention provides an emotional dialogue method for terminal digital humans based on a large model, which solves the problem of insufficient emotional consistency in the existing digital human dialogue system. Through the retrieval and generation mechanism of emotion perception, the generated dialogue can be more humane and effectively adapt to the user's emotional state, thereby improving the emotional interaction effect of the terminal digital human.

[0004] The present invention is achieved through the following technical solutions:

[0005] A method for emotional dialogue of a terminal digital human based on a large model is provided, comprising the following steps:

[0006] S1. Convert the user's input information into vector representations of semantics and emotions for subsequent retrieval. For query q, use the following formula to generate its semantic vector and emotional vector:

[0007] Semantic vector: semantic q =F(q);

[0008] Where F is the semantic embedding function, using the widely used pre-trained model BERT or bge-base-zh-v1.5, and each query is a 768-dimensional vector;

[0009] Emotion vector: emotion q =G(q);

[0010] Take query q as input and output its sentiment vector through the large model qwen32B; the output is an 8-dimensional vector containing 8 different sentiment states, and the value of each dimension is an integer between 1 and 10, which is used to measure the strength of the sentiment state. Use qwen32B to get the prompt of the vector;

[0011] S2, semantic and emotional encoding of each memory fragment stored in the system, the memory fragment is represented by m i ,

[0012] Semantic vector: semantic m =F(m i );

[0013] Emotion vector: emotion m =G(m i );

[0014] S3. By comparing the semantic and emotional similarity between the query and the memory fragment, the most suitable memory fragment is selected. The specific similarity calculation method is as follows:

[0015] Semantic similarity: score semantic =E(semantic q ,senantic m );

[0016] Sentiment similarity: score emotion =1-C(emotion q , emotion m );

[0017] Among them, C is the cosine similarity function of the emotion vector, which reflects the emotional consistency between the query and the memory fragment; E is the similarity function of the semantic vector, which reflects the semantic consistency between the query and the memory fragment;

[0018] The final retrieval score is score final =a*score semantic +(1-a)*score emotion ; Among them, a is a weight parameter, which can adjust the influence ratio of semantics and emotions;

[0019] S4. Based on the memory fragment with the highest retrieval score, the query, role information, and retrieved memory fragment information are input into the language model to generate a response with emotional resonance.

[0020] Furthermore, each question q contains a total of eight basic emotions, including joy, acceptance, fear, surprise, sadness, disgust, anger, and anticipation.

[0021] Further, output the result as a python list and parse it in python.

[0022] Beneficial effects of the present invention:

[0023] In order to solve the problem of insufficient emotional consistency in existing digital human dialogue systems, the present invention proposes a dialogue generation method that combines semantic and emotional information. Through the retrieval and generation mechanism of emotion perception, the generated dialogue can be more humane and effectively adapt to the user's emotional state, thus improving the emotional interaction effect of the terminal digital human.

[0024] The emotional dialogue method framework proposed in the present invention includes the following four core components: S1, query encoding component, S2, memory encoding component, S3, emotion retrieval component and S4, response generation component. Its workflow is as follows: after the user sends a query, the system retrieves the most matching memory fragment based on the semantic and emotional information of the query, so that when generating dialogue content, it conforms to the semantic logic and resonates with the user's emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is an overall schematic diagram of the present invention.

[0026] Figure 2 It is a basic information table of each data set in the present invention.

[0027] Figure 3 The following is a comparison table of the present invention and the traditional method. DETAILED DESCRIPTION

[0028] In order to clearly illustrate the technical features of this solution, this solution is described below through a specific implementation method.

[0029] An emotional dialogue method for a terminal digital human based on a large model includes the following steps:

[0030] S1. Convert the user's input information into vector representations of semantics and emotions for subsequent retrieval. For query q, use the following formula to generate its semantic vector and emotional vector:

[0031] Semantic vector: semantic q =F(q);

[0032] Where F is the semantic embedding function, using the widely used pre-trained model BERT or bge-base-zh-v1.5, and each query is a 768-dimensional vector;

[0033] Emotion vector: emotion q =G(q);

[0034] Take query q as input and output its sentiment vector through the large model qwen32B; the output is an 8-dimensional vector containing 8 different sentiment states (i.e., joy, acceptance, fear, surprise, sadness, disgust, anger, and expectation), and the value of each dimension is an integer between 1 and 10, which is used to measure the intensity of the sentiment state. Use qwen32B to get the prompt of the vector;

[0035] Output Format

[0036] The questions for each emotional dimension are analyzed and the results are output as a python list as shown below:

[0037] {"analysis":,"dimension":"joy","score":},{"analysis":,"dimension":"acceptance","score":}{"analysis":,"dimension":"expectation","score":}. This way you can parse it directly in Python.

[0038] S2, semantic and emotional encoding of each memory fragment stored in the system, the memory fragment is represented by m i ,

[0039] Semantic vector: semantic m =F(m i );

[0040] Emotion vector: emotion m =G(m i );

[0041] S3. By comparing the semantic and emotional similarity between the query and the memory fragment, the most suitable memory fragment is selected. The specific similarity calculation method is as follows:

[0042] Semantic similarity: score semantic =E(semantic q , semantic m );

[0043] Sentiment similarity: score emotion =1-C(emotion q , emotion m );

[0044] Among them, C is the cosine similarity function of the emotion vector, which reflects the emotional consistency between the query and the memory fragment; E is the similarity function of the semantic vector, which reflects the semantic consistency between the query and the memory fragment;

[0045] The final search score is:

[0046] score final =a*scoresemantic +(1-a)*score emotion ;

[0047] Among them, a is a weight parameter, which can adjust the influence ratio of semantics and emotions;

[0048] S4. Based on the memory fragment with the highest retrieval score, the query, role information, and retrieved memory fragment information are input into the language model to generate a response with emotional resonance.

[0049] The method of the present invention is experimentally designed, in which the experimental data set uses three public data sets: InCharacter, CharacterEval and Character-LLM. Each data set contains rich role-playing dialogue data, providing a verification environment for emotional dialogue generation in different scenarios. The basic information of each data set is as follows Figure 2 shown.

[0050] In order to evaluate the emotional consistency and personality trait performance, the Big Five Personality Inventory (BFI) and the MBTI personality test were used. Specific indicators include: (1) Accuracy: divided into dimensional accuracy (Acc(Dim)) and full dimensional accuracy (Acc(Full)). (2) Mean square error (MSE) and mean absolute error (MAE): measure the deviation between the predicted personality trait value and the true value.

[0051] The experimental results are as follows:

[0052] like Figure 3 As shown, the performance of the present invention on the InCharacter dataset is demonstrated, indicating that the generation effect after introducing emotion retrieval is significantly better than the traditional RAG method.

[0053] The present invention introduces an emotional retrieval mechanism and designs an innovative emotional retrieval enhancement generation method by combining semantic and emotional information, so that the generated dialogue content conforms to semantic logic and has emotional consistency.

[0054] In the field of mental health, emotional responses are crucial for psychological counseling and emotional support. Based on the emotional retrieval and generation technology of the present invention, the system can identify the user's emotional state based on the user's input content, retrieve emotionally consistent cases or coping methods from the memory bank, and thus generate a more empathetic and supportive response. Emotionally consistent responses can make users feel understood and supported, help build trust, and increase users' dependence on and satisfaction with the system, especially in dealing with emotions such as depression and anxiety.

[0055] In intelligent education, different students may have different emotional states, such as frustration, anxiety, or joy. The system can identify students' learning states through emotion perception, so as to select appropriate teaching content and methods, and provide motivational feedback that matches emotions. Emotion perception and emotion-consistent feedback can make students feel understood and encouraged during the learning process, helping them maintain a positive learning mood. This personalized feedback can enhance the learning experience and effect, and is particularly suitable for adaptive learning systems and online education platforms.

[0056] Of course, the above description is not limited to the above examples. Technical features not described in the present invention can be achieved by or by using existing technologies, which will not be described here. The above embodiments and drawings are only used to illustrate the technical scheme of the present invention and are not limitations of the present invention. The present invention is described in detail with reference to the preferred implementation methods. Ordinary technicians in this field should understand that changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention do not depart from the purpose of the present invention, and should also fall within the scope of protection of the claims of the present invention.

Claims

1. An emotional dialogue method for a terminal digital human based on a large model, characterized by: The following steps are involved: S1. Convert the user's input information into vector representations of semantics and emotions for subsequent retrieval. For query q, use the following formula to generate its semantic vector and emotional vector: Semantic vector: semantic q =F(q); Where F is the semantic embedding function, using the widely used pre-trained model BERT or bge-base-zh-v1.5, and each query is a 768-dimensional vector; Emotion vector: emotion q =G(q); Take query q as input and output its sentiment vector through the large model qwen32B; the output is an 8-dimensional vector containing 8 different sentiment states, and the value of each dimension is an integer between 1 and 10, which is used to measure the strength of the sentiment state. Use qwen32B to get the prompt of the vector; S2, semantic and emotional encoding of each memory fragment stored in the system, the memory fragment is represented by m i , Semantic vector: semantic m =F(m i ); Emotion vector: emotion m =G(m i ); S3. By comparing the semantic and emotional similarity between the query and the memory fragment, the most suitable memory fragment is selected. The specific similarity calculation method is as follows: Semantic similarity: score semantic =E(semantic q , semantic m ); Sentiment similarity: score emotion =1-C(emotion q , emotion m ); Among them, C is the cosine similarity function of the emotion vector, which reflects the emotional consistency between the query and the memory fragment; E is the similarity function of the semantic vector, which reflects the semantic consistency between the query and the memory fragment; The final retrieval score is score final =a*score semantic +(1-a)*score emotion ; Among them, a is a weight parameter, which can adjust the influence ratio of semantics and emotions; S4. Based on the memory fragment with the highest retrieval score, the query, role information, and retrieved memory fragment information are input into the language model to generate a response with emotional resonance.

2. The emotional dialogue method of terminal digital human based on large model according to claim 1 is characterized by: In step S1, each question q contains a total of eight basic emotions, including joy, acceptance, fear, surprise, sadness, disgust, anger, and anticipation.

3. The emotional dialogue method of terminal digital human based on large model according to claim 2 is characterized by: In step S1, output the result as a python list, and parse it in python.

Images