Digital human generation method and device, and electronic device

Abstract

The application discloses a digital human generation method and device and electronic equipment, and belongs to the field of image processing. The method comprises the following steps: acquiring a first dialogue data set, wherein the first dialogue data set comprises M questions and a first answer corresponding to each question and carrying a target emotion label; training a large language model through the first dialogue data set to obtain a first target large language model, wherein the first target large language model can output an answer carrying emotion; acquiring a target personality category, language habit and basic information of a character; inputting the target personality category, the language habit and the basic information of the character into the first target large language model to obtain a second target large language model, wherein the second target large language model can simulate the personality and language habit of a person to output an answer carrying emotion; and associating the second target large language model with a digital human to obtain a target digital human corresponding to the second target large language model.

Description

Technical Field

[0001] This application belongs to the field of artificial intelligence technology, specifically relating to a method, apparatus, and electronic device for generating digital humans. Background Technology

[0002] Current optimizations of digital human technology generally focus on improving the realism and naturalness of digital humans. Realism includes enhancing facial expressions and subtle movements, while naturalness primarily involves improving the fluency and similarity of speech. However, because digital humans have not yet mastered emotional understanding and expression, their effectiveness and anthropomorphism in practical applications are limited, resulting in a relatively low level of intelligence among existing digital humans. Summary of the Invention

[0003] The purpose of this application is to provide a digital human generation method, apparatus, and electronic device that can solve the problem of low intelligence caused by digital humans' lack of emotional understanding and expression.

[0004] In a first aspect, embodiments of this application provide a digital human generation method, including:

[0005] Obtain the first dialogue dataset, which includes M questions and the first answer for each question carrying the target sentiment label, where M is an integer greater than 1;

[0006] The first target large language model is obtained by training the large language model using the first dialogue dataset. The first target large language model can output answers that carry emotion.

[0007] Obtain the target's personality type, language habits, and basic information about the person;

[0008] The target personality category, the language habits, and the basic information of the person are input into the first target large language model to obtain the second target large language model. The second target large language model can simulate a person's personality and language habits to output answers that carry emotions.

[0009] By associating the second target large language model with the digital human, a target digital human corresponding to the second target large language model is obtained.

[0010] Secondly, embodiments of this application provide a digital human generation device, comprising:

[0011] The first acquisition module is used to acquire the first dialogue dataset, which includes M questions and the first answer carrying the target sentiment label for each question, where M is an integer greater than 1.

[0012] The training module is used to train the large language model using the first dialogue dataset to obtain a first target large language model, which can output answers carrying emotions.

[0013] The second acquisition module is used to acquire the target's personality type, language habits, and basic information about the person.

[0014] The processing module is used to input the target personality category, the language habits and the basic information of the person into the first target large language model to obtain the second target large language model. The second target large language model can simulate a person's personality and language habits to output answers that carry emotions.

[0015] The association module is used to associate the second target large language model with the digital human to obtain the target digital human corresponding to the second target large language model.

[0016] Thirdly, embodiments of this application also provide an electronic device, including a memory, a transceiver, and a processor:

[0017] A memory for storing computer programs; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer programs from the memory and executing the method described in the first aspect above.

[0018] Fourthly, embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to perform the method described in the first aspect above.

[0019] In this embodiment, a first dialogue dataset is acquired, comprising M questions and a first answer carrying a target sentiment label for each question. A large language model is trained using this dataset to obtain a first target large language model. This first target large language model can output answers carrying sentiment. Thus, when a user inputs a question, the first target large language model can output an answer with sentiment. Furthermore, by acquiring target personality categories, language habits, and basic character information, and inputting these into the first target large language model, a second target large language model is obtained. This second target large language model can simulate human personality and language habits to output answers carrying sentiment. The second target large language model is then associated with a digital human to obtain a target digital human corresponding to the second target large language model. Therefore, when a user inputs a question, the second target large language model can simulate human personality and language habits to output an answer carrying sentiment. Furthermore, the expression by the target digital human makes the dialogue between the user and the target digital human more intelligent and realistic, improving the user experience. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating the digital human generation method provided in an embodiment of this application;

[0021] Figure 2 This is a schematic diagram of the input and output of the second target large language model provided in the embodiments of this application;

[0022] Figure 3 This is a schematic diagram of the annotation of emotion tags provided in the embodiments of this application;

[0023] Figure 4 This is a schematic diagram illustrating the training of the target sentiment model provided in an embodiment of this application;

[0024] Figure 5 This is a schematic diagram illustrating the generation of the first judgment result provided in an embodiment of this application;

[0025] Figure 6 This is a schematic diagram of the dialogue vector data and prototype vector provided in the embodiments of this application;

[0026] Figure 7 This is a schematic diagram of the structure of the digital human generation device provided in the embodiments of this application;

[0027] Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0028] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0029] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0030] The digital human generation method provided in this application will be described below with reference to the accompanying drawings, through specific embodiments and application scenarios.

[0031] like Figure 1 As shown in the figure, this application provides a method for generating a digital human, which may specifically include the following steps:

[0032] Step 101: Obtain the first dialogue dataset, which includes M questions and the first answer carrying the target sentiment label for each question, where M is an integer greater than 1.

[0033] Specifically, we obtain M questions and the first answer for each of the M questions. Each question and its corresponding first answer constitutes one dialogue data point. The M questions and M first answers constitute M dialogue data points, and the M dialogue data points together form the first dialogue dataset. Each of the M first answers carries a target sentiment label.

[0034] Among them, the target emotional labels include, but are not limited to: happiness label, anger label, disappointment label, fear label, surprise label, and no emotion label.

[0035] Step 102: Train the large language model using the first dialogue dataset to obtain a first target large language model, which can output answers carrying emotions.

[0036] Specifically, the first dialogue dataset is used as the sample dataset to train the large language model. Each question from the first dialogue dataset is input into the large language model. Based on the large language model's text understanding and reasoning capabilities, the answer to each question is obtained. The model parameters are continuously optimized by comparing the answers obtained from the large language model with the initial answer, until the large language model reaches convergence, resulting in the first target large language model. Inputting a user question into the first target large language model yields an answer that carries emotional weight.

[0037] Step 103: Obtain the target's personality type, language habits, and basic information about the person.

[0038] Furthermore, the basic information of the person includes, but is not limited to, at least one of the following: identity information, personal experience information, values ​​information, and social relationship information.

[0039] Specifically, characteristics such as the target's personality type, language habits, and basic information are obtained, and these characteristics are mapped to a personalized table of the digital person, as shown in Table 1:

[0040] Table 1

[0041]

[0042] Herein, the language habits may include, but are not limited to, pet phrases (e.g., "You are great", "Oh my"), dialect terms (e.g., "do"), commonly used phrases (e.g., "hehe", "haha"), language style features (e.g., "formal language style feature"), and the like. The basic personal information may include, but is not limited to, identity information, personal experience information, three-view information, social relationship information, and the like. The identity information may include name, gender, age, interests, etc.; the personal experience information may include a brief biography of the life, important events experienced, etc.; the three-view information may include world view, life view, value view, etc., and the value view may be dialectical materialism, including views and thoughts on certain specific events.

[0043] Step 104: input the target personality category, the language habits and the basic personal information into the first target large language model to obtain a second target large language model, where the second target large language model can simulate human personality and language habits to output answers with emotions.

[0044] Specifically, the target personality category, the language habits and the basic personal information are input into the first target large language model, and the first target large language model is packaged through the target personality category, the language habits and the basic personal information to obtain the second target large language model. The second large language model can simulate human personality, language habits, three views, etc., and generate answers with emotions for the input questions. For example, as Figure 2 shown, the user's question "Are you happy today" is input into the second target large language model, and the emotionally charged answer "I am very happy today [Happy]" is obtained, where [Happy] represents a happiness tag.

[0045] Step 105: associate the second target large language model with a digital human to obtain a target digital human corresponding to the second target large language model.

[0046] Specifically, the second target large language model is associated with a digital human to obtain a target digital human corresponding to the second target large language model. This target digital human can simulate a person's personality, language habits, values, etc., meaning it is essentially a simulated human with personality, language habits, and values. If a user's voice question is received, speech-to-text technology converts the voice question into a text question, which is then input into the second target large language model. The second target large language model answers the question based on personality, language habits, and values, resulting in an emotional text answer. Text-to-speech technology converts the text answer into a voice answer, which is then expressed by the target digital human. That is, through facial and body driving modules, the target digital human is driven to output corresponding voice, facial expressions, and gestures based on the voice answer, allowing the user to understand the emotion behind the answer. Moreover, the user can learn about the target digital human's personality, language habits, values, etc., through the answer expressed by the target digital human.

[0047] The process of building a digital human will be explained in detail below:

[0048] First, the appearance, characteristics, and personality of the digital human need to be determined, including facial expressions, body proportions, and clothing. Then, computer software is used to model and sculpt the digital human in detail. Modeling refers to creating the basic shape and structure of the digital human, while sculpting adds more details (such as skin texture, muscle lines, hair, and clothing) to the basic model to make it more realistic. Adding a skeleton and animation to the digital human is key to giving it movement and expression. Through skeletal rigging and animation control techniques, the digital human model can have movable joints and natural movements. Next, by adjusting lighting settings and rendering parameters, the appearance of the digital human can be matched with the environment, producing realistic shadows, reflections, and refractions. Finally, post-production and special effects are applied to the digital human, including color correction, special effects, and compositing, to further improve the visual quality and realism. A second target large language model is bound and associated with the digital human to obtain a target digital human corresponding to the second target large language model. This target digital human is used to simulate a human to express the answers output by the second target large language model. In this embodiment, a first dialogue dataset is acquired, comprising M questions and a first answer carrying a target sentiment label for each question. A large language model is trained using this dataset to obtain a first target large language model. This first target large language model can output answers carrying sentiment. Thus, when a user inputs a question, the first target large language model can output an answer with sentiment. Furthermore, by acquiring target personality categories, language habits, and basic character information, and inputting these into the first target large language model, a second target large language model is obtained. This second target large language model can simulate human personality and language habits to output answers carrying sentiment. The second target large language model is then associated with a digital human to obtain a target digital human corresponding to the second target large language model. Therefore, when a user inputs a question, the second target large language model can simulate human personality and language habits to output an answer carrying sentiment. Furthermore, the expression by the target digital human makes the dialogue between the user and the target digital human more intelligent and realistic, improving the user experience.

[0049] As an optional specific embodiment, step 101, which involves obtaining the first dialogue dataset, specifically includes steps 1011 to 1014:

[0050] Step 1011: Obtain M questions and M second answers corresponding to the M questions from the second dialogue dataset;

[0051] Step 1012: Input the i-th second answer among the M second answers into the multi-label classification model to obtain the N sentiment labels corresponding to the i-th second answer and the probability value corresponding to each sentiment label. i takes the value of each integer from 1 to M, and N is an integer greater than 0.

[0052] Step 1013: Based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first value, determine the target sentiment tag of the i-th second answer, so as to determine the target sentiment tags of the M second answers;

[0053] Step 1014: Add the corresponding target sentiment tag to the M second answers to obtain the first answer carrying the target sentiment tag for each of the M questions.

[0054] Specifically, we obtain M questions and the second answer for each of the M questions. Each question and its corresponding second answer constitutes a dialogue data set. The M questions and their M second answers constitute M dialogue data sets. The M dialogue data sets together form the second dialogue dataset.

[0055] The following explanation uses the i-th second answer out of M second answers:

[0056] The i-th second answer is input into a multi-label classification model to obtain N sentiment labels corresponding to the i-th second answer and the probability value of each sentiment label, such as: {Happy label: 0.5, Sad label: 0.4, Surprised label: 0.1, Angry label: 0.3, Fear label: 0.1, No sentiment label: 0.1}. Since the N sentiment labels and the probability value of each sentiment label are obtained by the multi-label classification model, the sum of the N probability values ​​can be greater than 1.

[0057] Compare the probability value corresponding to each sentiment tag of the i-th second answer with the first value (e.g., 0.5) to obtain the relationship between the probability value corresponding to each sentiment tag and the first value. The target sentiment tag of the i-th second answer can be determined by the relationship between the probability value corresponding to each sentiment tag and the first value.

[0058] It should be noted that the target sentiment label of each of the M second answers is determined according to the process of the target sentiment label of the i-th second answer described above, thereby determining the target sentiment label of each of the M second answers.

[0059] Add its corresponding target sentiment label to each of the M second answers, thereby obtaining the first answer carrying the target sentiment label for each second answer. The M questions and the M first answers corresponding to the M questions constitute the first dialogue dataset.

[0060] As an optional specific embodiment, step 1013, based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first numerical value, determines the target sentiment tag of the i-th second answer, specifically including:

[0061] If all N probability values ​​corresponding to the N sentiment tags of the i-th second answer are less than or equal to the first value, or if the N probability values ​​include a target probability value and the sentiment tag corresponding to the target probability value is without a sentiment tag, the i-th second answer carrying the j-th sentiment tag from the N sentiment tags is input into the multiple target sentiment models corresponding to the j-th sentiment tag to obtain multiple first judgment results for the j-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn;

[0062] Based on the first determination rule, the second determination result of the j-th emotion tag is determined through multiple first determination results of the j-th emotion tag, so as to obtain N second determination results of the N emotion tags;

[0063] Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the N second judgment results of the N sentiment labels.

[0064] Specifically, the probability value corresponding to each sentiment tag of the i-th second answer is compared with the first value. If all N probability values ​​corresponding to the N sentiment tags of the i-th second answer are less than or equal to the first value, or if at least one of the N probability values ​​is greater than the target probability value of the first value and the sentiment tags corresponding to the target probability values ​​are all no sentiment tags, then the target sentiment tag of the i-th second answer is determined according to the following rules:

[0065] The process involves iterating through the target sentiment models corresponding to the i-th second answer carrying different sentiment tags, obtaining the first judgment result for each target sentiment model. Specifically, the i-th second answer carrying the first sentiment tag is input into each of the multiple target sentiment models corresponding to the first sentiment tag, resulting in the first judgment result output by each model. This process is repeated until the i-th second answer carrying the N-th sentiment tag is input into each of the multiple target sentiment models corresponding to the N-th sentiment tag, resulting in the first judgment result output by each model.

[0066] For example: the first value is 0.5, N is 4, and there are 4 emotion labels and their corresponding 4 probability values: {Happy label: 0.4, Sad label: 0.4, Surprised label: 0.1, Angry label: 0.3}. Since all 4 probability values ​​are less than or equal to 0.5, the i-th second answer carrying the happy label is input into multiple happy target emotion models (i.e., multiple target emotion models corresponding to the happy label), and the first judgment result output by the multiple happy target emotion models is obtained; then the i-th second answer carrying the sad label is input into multiple sad target emotion models (i.e., multiple target emotion models corresponding to the sad label), and the first judgment result output by the multiple sad target emotion models is obtained; then the i-th second answer carrying the surprise label is input into multiple surprised target emotion models (i.e., multiple target emotion models corresponding to the surprise label), and the first judgment result output by the multiple surprised target emotion models is obtained; then the i-th second answer carrying the anger label is input into multiple angry target emotion models (i.e., multiple target emotion models corresponding to the anger label), and the first judgment result output by the multiple angry target emotion models is obtained.

[0067] It should be noted that each sentiment tag corresponds to multiple different target sentiment models, that is, each sentiment tag corresponds to multiple (e.g., 3) different target sentiment models, which are based on statistical feature classification models, word classification models, semantic classification models, etc. Multiple different target sentiment models can judge the correctness of sentiment tags from multiple dimensions. Among them, the target sentiment model is a sentiment binary classification model.

[0068] The first judgment results of the j-th sentiment label are used to determine the second judgment result of the j-th sentiment label according to the first determination rule, so as to obtain the second judgment result of each sentiment label in the N sentiment labels; the N second judgment results of the N sentiment labels are used to determine the target sentiment label of the i-th second answer according to the second determination rule.

[0069] The first determination rule is explained in detail below:

[0070] For multiple first judgment results of a sentiment label, those with the same first judgment result are grouped into one category for statistical analysis.

[0071] Based on the first judgment result of the category with the most occurrences, the second judgment result of the sentiment label is determined.

[0072] Specifically, if multiple first judgment results for the j-th sentiment label have the same result, these identical first judgment results are grouped into one category, and the number of first judgment results in the same category is counted. The number of first judgment results in different categories is compared, and the second judgment result for the j-th sentiment label is determined based on the category with the most first judgment results.

[0073] For example, the "happy" tag has three first judgment results: "is a happy tag", "is not a happy tag", and "is a happy tag". Since there are two first judgment results for "is a happy tag" and one for "is not a happy tag", the second judgment result for the "happy tag" needs to be determined based on the first judgment result for "is a happy tag".

[0074] The second determination rule will be explained in detail below:

[0075] For multiple second judgment results of multiple sentiment tags, if the multiple second judgment results are all the same, then the target sentiment tag is determined to be the sentiment tag corresponding to the second judgment result; otherwise, according to the priority order of sentiment tags, the target sentiment tag is determined to be the sentiment tag with the highest priority among the multiple second judgment results.

[0076] Specifically, for N second judgment results corresponding to N sentiment tags, if all N second judgment results are the same, then the sentiment tag corresponding to the second judgment result is the target sentiment tag. If not all N second judgment results are the same, then according to the priority order of sentiment tags, the sentiment tag with the highest priority among the N sentiment tags corresponding to the N second judgment results is determined as the target sentiment tag.

[0077] For example, the second judgment result for the "happy" label is "it is a happy label," the second judgment result for the "angry" label is "not an angry label," and the second judgment result for the "surprised" label is "it is a surprised label." Since the negative result "not an angry label" only indicates that it is not an angry label and does not specify the emotional label, we can replace all negative results like "not an angry label" with an "emotionless label." That is, the second judgment result for the "happy" label is "it is a happy label," the second judgment result for the "angry" label is "emotionless label," and the second judgment result for the "surprised" label is "it is a surprised label." Because the three second judgment results are different, we use the "surprised" label as the target emotional label according to the priority order of emotional labels (fear label > anger label > surprise label > sadness label > happy label > emotionless label).

[0078] The training process of the above-mentioned target emotion model is illustrated below through a specific embodiment:

[0079] like Figure 3 As shown, firstly, a dialogue is conducted between the user and a large language model to obtain multi-turn dialogue text. This text is then labeled with sentiment tags, resulting in dialogue text with sentiment labels such as "happy," "angry," and "fearful," thus imbuing the dialogue text with emotional information (e.g., happiness, anger, fear). Only a small amount of labeling is required. Secondly, leveraging data augmentation capabilities, the existing small amount of labeled data is expanded. Then, based on the expanded data, multiple sentiment binary classification models are trained (each sentiment corresponds to multiple different sentiment binary classification models), resulting in multiple trained target sentiment models. For example: Figure 4 As shown, firstly, data with expanded "happiness" labels are obtained, and then multiple binary sentiment classification models are trained based on the expanded "happiness" label data to obtain three target sentiment models corresponding to the "happiness" labels, namely, target sentiment model 1 for "happiness", target sentiment model 2 for "happiness", and target sentiment model 3 for "happiness".

[0080] As an optional specific embodiment, step 1013, which determines the target sentiment label of the i-th second answer based on the relationship between the probability value corresponding to each sentiment label of the i-th second answer and the first numerical value, further includes:

[0081] If there are P target probability values ​​among the N probability values, and not all of the P sentiment tags corresponding to the P target probability values ​​are without sentiment tags, the i-th second answer carrying the k-th sentiment tag among the P sentiment tags is input into the multiple target sentiment models corresponding to the k-th sentiment tag to obtain multiple first judgment results for the k-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, P is an integer greater than 0 and less than or equal to N, and k takes the value of each integer from 1 to P in sequence;

[0082] Based on the first determination rule, the second determination result of the kth emotion tag is determined through multiple first determination results of the kth emotion tag, so as to obtain P second determination results of the P emotion tags;

[0083] Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the P second judgment results of the P sentiment labels.

[0084] Specifically, the probability value corresponding to each sentiment tag of the i-th second answer is compared with the first value. If P of the N probability values ​​corresponding to the N sentiment tags of the i-th second answer are greater than the first value, and among the P sentiment tags corresponding to the P target probability values, there is one that is not without a sentiment tag, then the target sentiment tag of the i-th second answer is determined according to the following rules:

[0085] The process involves iterating through the target sentiment models corresponding to each of the P sentiment tags using the i-th second answer carrying different sentiment tags. This yields a first judgment result for each target sentiment model. Specifically, the i-th second answer carrying the first sentiment tag from the P sentiment tags is input into each of the multiple target sentiment models corresponding to the first sentiment tag from the P sentiment tags, resulting in the first judgment result output by each target sentiment model. This process is repeated until the i-th second answer carrying the P-th sentiment tag from the P sentiment tags is input into each of the multiple target sentiment models corresponding to the P-th sentiment tag from the P sentiment tags, resulting in the first judgment result output by each target sentiment model.

[0086] For example: Figure 5As shown, the first value is 0.5. First, the i-th second answer is obtained, and then it is input into the multi-label classification model to obtain the four sentiment labels corresponding to the i-th second answer and the four probability values ​​corresponding to these four sentiment labels. The four sentiment labels and their corresponding four probability values ​​are: {Happy label: 0.5, Sad label: 0.4, Surprised label: 0.1, Angry label: 0.6}. Since the probability values ​​of the Happy label and the Angry label are greater than 0.5, the target probability value for the Happy label is 0.5, and the target probability value for the Angry label is 0.6. The i-th second answer carrying the "happiness" label is input into the target sentiment models A1, A2, and A3 for happiness, respectively, resulting in the first judgment result "is happy" for target sentiment model A1, "is happy" for target sentiment model A2, and "not happy" for target sentiment model A3. Then, the i-th second answer carrying the "anger" label is input into the target sentiment models B1, B2, and B3 for anger, resulting in the first judgment result "is angry" for target sentiment model B1, "is angry" for target sentiment model B2, and "not angry" for target sentiment model B3. This saves computational resources for sentiment labels with lower probability values, improving efficiency.

[0087] The first judgment results of the k-th sentiment label are used to determine the second judgment result of the k-th sentiment label according to the first determination rule, so as to obtain the second judgment result of each sentiment label in P sentiment labels; the P second judgment results of P sentiment labels are used to determine the target sentiment label of the i-th second answer according to the second determination rule.

[0088] The first determination rule is explained in detail below:

[0089] For multiple first judgment results of a sentiment label, those with the same first judgment result are grouped into one category for statistical analysis.

[0090] Based on the first judgment result of the category with the most occurrences, the second judgment result of the sentiment label is determined.

[0091] Specifically, if multiple first judgment results for the k-th sentiment label have the same result, these identical first judgment results are grouped into one category, and the number of first judgment results in the same category is counted. The number of first judgment results in different categories is compared, and the second judgment result for the k-th sentiment label is determined based on the category with the most first judgment results.

[0092] For example, the "happy" tag has three first judgment results: "is a happy tag", "is not a happy tag", and "is a happy tag". Since there are two first judgment results for "is a happy tag" and one for "is not a happy tag", the second judgment result for the "happy tag" needs to be determined based on the first judgment result for "is a happy tag".

[0093] The second determination rule will be explained in detail below:

[0094] For multiple second judgment results of multiple sentiment tags, if the multiple second judgment results are all the same, then the target sentiment tag is determined to be the sentiment tag corresponding to the second judgment result; otherwise, according to the priority order of sentiment tags, the target sentiment tag is determined to be the sentiment tag with the highest priority among the multiple second judgment results.

[0095] Specifically, for P sentiment labels corresponding to P second judgment results, if all P second judgment results are the same, then the sentiment label corresponding to the second judgment result is the target sentiment label. If not all P second judgment results are the same, then according to the priority order of sentiment labels, the sentiment label with the highest priority among the P sentiment labels corresponding to the P second judgment results is determined as the target sentiment label.

[0096] For example, the second judgment result for the "happy" label is "it is a happy label," the second judgment result for the "angry" label is "not an angry label," and the second judgment result for the "surprised" label is "it is a surprised label." Since the negative result "not an angry label" only indicates that it is not an angry label and does not specify the emotional label, we can replace all negative results like "not an angry label" with an "emotionless label." That is, the second judgment result for the "happy" label is "it is a happy label," the second judgment result for the "angry" label is "emotionless label," and the second judgment result for the "surprised" label is "it is a surprised label." Because the three second judgment results are different, we use the "surprised" label as the target emotional label according to the priority order of emotional labels (fear label > anger label > surprise label > sadness label > happy label > emotionless label).

[0097] As an optional specific embodiment, step 103, obtaining the target personality, specifically includes:

[0098] Obtain the prototype vector of each personality category from multiple personality categories, as well as the target person's historical dialogue data;

[0099] The historical dialogue data is vectorized using a prototype network to obtain dialogue vector data.

[0100] The prototype vector of each personality category is compared with the dialogue vector data to determine the personality category corresponding to the prototype vector with the highest similarity.

[0101] Specifically, such as Figure 6 As shown, based on Prototypical Networks (PN), the 16 personality types (such as Scientist, Scholar, Commander, etc.) in the Myers-Briggs Type Indicator (MBTI) are learned to obtain the prototype vectors of the corresponding personality types, such as... Figure 6 The prototype vectors shown are for the scientist type, scholar type, commander type, etc. To define a person's personality type, given multiple rounds (e.g., 10 rounds or more) of historical dialogue data, the corresponding prototype network is used to vectorize the historical dialogue data to obtain dialogue vector data. The dialogue vector data is then compared with the prototype vectors of each personality type (i.e., similarity comparison), and the personality type corresponding to the prototype vector with the closest distance (i.e., the highest similarity) is selected as the target personality type.

[0102] In summary, this embodiment trains a large language model using M questions and a first answer carrying a target emotional label for each question, resulting in a first target large language model capable of outputting emotionally charged answers. Thus, when a user inputs a question, the first target large language model can output an emotionally charged answer. Furthermore, by inputting target personality categories, language habits, and basic character information into the first target large language model for packaging, a second target large language model is obtained that can simulate human personality and language habits to output emotionally charged answers. This second target large language model is then associated with a digital human to obtain a target digital human corresponding to the second target large language model. Therefore, after a user inputs a question, the second target large language model can simulate human personality, language habits, and values ​​to output emotionally charged answers. Moreover, the expression by the target digital human makes the dialogue between the user and the target digital human more intelligent and realistic, improving the user experience.

[0103] The digital human generation method provided in this application can be executed by a digital human generation device. This application uses an example of a digital human generation device executing the digital human generation method to illustrate the digital human generation device provided in this application.

[0104] like Figure 7 As shown in the illustration, this application also provides a digital human generation device 700, which specifically includes:

[0105] The first acquisition module 701 is used to acquire a first dialogue dataset, which includes M questions and a first answer carrying a target sentiment label for each question, where M is an integer greater than 1.

[0106] Training module 702 is used to train the large language model using the first dialogue dataset to obtain a first target large language model, which can output answers carrying emotions.

[0107] The second acquisition module 703 is used to acquire the target personality type, language habits and basic information about the person.

[0108] Processing module 704 is used to input the target personality category, the language habits and the basic information of the person into the first target large language model to obtain a second target large language model. The second target large language model can simulate a person's personality and language habits to output answers carrying emotions.

[0109] The association module 705 is used to associate the second target large language model with the digital human to obtain the target digital human corresponding to the second target large language model.

[0110] As an optional specific embodiment, the first acquisition module 701 is specifically used for:

[0111] Obtain M questions from the second dialogue dataset and M second answers corresponding to the M questions;

[0112] The i-th second answer among the M second answers is input into the multi-label classification model to obtain the N sentiment labels corresponding to the i-th second answer and the probability value corresponding to each sentiment label. i takes the value of each integer from 1 to M, and N is an integer greater than 0.

[0113] Based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first value, the target sentiment tag of the i-th second answer is determined, thereby determining the target sentiment tags of the M second answers;

[0114] Add the corresponding target sentiment label to the M second answers to obtain the first answer carrying the target sentiment label for each of the M questions.

[0115] As an optional specific embodiment, when the first acquisition module 701 determines the target sentiment tag of the i-th second answer based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first numerical value, it is specifically used for:

[0116] If all N probability values ​​corresponding to the N sentiment tags of the i-th second answer are less than or equal to the first value, or if the N probability values ​​include a target probability value and the sentiment tag corresponding to the target probability value is without a sentiment tag, the i-th second answer carrying the j-th sentiment tag from the N sentiment tags is input into the multiple target sentiment models corresponding to the j-th sentiment tag to obtain multiple first judgment results for the j-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn;

[0117] Based on the first determination rule, the second determination result of the j-th emotion tag is determined through multiple first determination results of the j-th emotion tag, so as to obtain N second determination results of the N emotion tags;

[0118] Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the N second judgment results of the N sentiment labels.

[0119] As an optional specific embodiment, when the first acquisition module 701 determines the target sentiment tag of the i-th second answer based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first numerical value, it is further configured to:

[0120] If there are P target probability values ​​among the N probability values, and not all of the P sentiment tags corresponding to the P target probability values ​​are without sentiment tags, the i-th second answer carrying the k-th sentiment tag among the P sentiment tags is input into the multiple target sentiment models corresponding to the k-th sentiment tag to obtain multiple first judgment results for the k-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, P is an integer greater than 0 and less than or equal to N, and k takes the value of each integer from 1 to P in sequence;

[0121] Based on the first determination rule, the second determination result of the kth emotion tag is determined through multiple first determination results of the kth emotion tag, so as to obtain P second determination results of the P emotion tags;

[0122] Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the P second judgment results of the P sentiment labels.

[0123] As an optional specific embodiment, the first determining rule includes:

[0124] For multiple first judgment results of a sentiment label, those with the same first judgment result are grouped into one category for statistical analysis.

[0125] Based on the first judgment result of the category with the most occurrences, the second judgment result of the sentiment label is determined.

[0126] As an optional specific embodiment, the second determining rule includes:

[0127] For multiple second judgment results of multiple sentiment tags, if the multiple second judgment results are all the same, then the target sentiment tag is determined to be the sentiment tag corresponding to the second judgment result; otherwise, according to the priority order of sentiment tags, the target sentiment tag is determined to be the sentiment tag with the highest priority among the multiple second judgment results.

[0128] As an optional specific embodiment, the second acquisition module 703, when acquiring the target personality, is specifically used for:

[0129] Obtain the prototype vector of each personality category from multiple personality categories, as well as the target person's historical dialogue data;

[0130] The historical dialogue data is vectorized using a prototype network to obtain dialogue vector data.

[0131] The prototype vector of each personality category is compared with the dialogue vector data to determine the personality category corresponding to the prototype vector with the highest similarity.

[0132] As an optional specific embodiment, the basic information of the person includes at least one of the following: identity information, personal experience information, values ​​information, and social relationship information.

[0133] It should be noted that the digital human generation apparatus provided in this application embodiment can implement all the method steps implemented in the above digital human generation method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0134] It should be noted that the division of units in the embodiments of this application is illustrative and only represents one logical functional division. In actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated units described above can be implemented in hardware or as software functional units.

[0135] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0136] like Figure 8 As shown, embodiments of this application also provide an electronic device, including a memory 820, a transceiver 810, and a processor 800:

[0137] Memory 820 is used to store computer programs;

[0138] Transceiver 810 is used to send and receive data under the control of the processor;

[0139] Processor 800 is configured to read a computer program from memory and execute the steps of the digital human generation method as described in any of the above embodiments.

[0140] Among them, Figure 8 In this context, the bus architecture can include any number of interconnected buses and bridges, specifically linking various circuits together, represented by one or more processors (processor 800) and memory (memory 820). The bus architecture can also link various other circuits such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. The bus interface provides an interface. The transceiver 810 can be multiple elements, including transmitters and receivers, providing a unit for communicating with various other devices over transmission media, including wireless channels, wired channels, optical fibers, etc. The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 can store data used by the processor 800 during operation.

[0141] The processor 800 can be a central processing unit (CPU), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a complex programmable logic device (CPLD). The processor can also adopt a multi-core architecture.

[0142] The processor executes the digital human generation method provided in this application embodiment by calling a computer program stored in memory, according to the obtained executable instructions. The processor and memory can also be physically separated.

[0143] It should be noted that the electronic device provided in this application embodiment can implement all the method steps implemented in the above-mentioned digital human generation method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0144] Embodiments of this application also provide a processor-readable storage medium storing a computer program for causing the processor to execute the above-described digital human generation method.

[0145] The processor-readable storage medium can be any available medium or data storage device that the processor can access, including but not limited to magnetic memory (e.g., floppy disk, hard disk, magnetic tape, magneto-optical disk (MO)), optical memory (e.g., CD, DVD, BD, HVD), and semiconductor memory (e.g., ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drive (SSD)).

[0146] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0147] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-executable instructions. These computer-executable instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0148] These processor-executable instructions may also be stored in a processor-readable memory that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the processor-readable memory produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0149] These processors can execute instructions that can also be loaded onto a computer or other programmable data processing device, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0150] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. A method for generating a digital human, characterized in that, include: Obtain the first dialogue dataset, which includes M questions and the first answer for each question carrying the target sentiment label, where M is an integer greater than 1; The first target large language model is obtained by training the large language model using the first dialogue dataset. The first target large language model can output answers that carry emotion. Obtain the target's personality type, language habits, and basic information about the person; The target personality category, the language habits, and the basic information of the person are input into the first target large language model to obtain the second target large language model. The second target large language model can simulate a person's personality and language habits to output answers that carry emotions. The second target large language model is associated with the digital human to obtain the target digital human corresponding to the second target large language model; The process of obtaining the first dialogue dataset includes: Obtain the N sentiment labels of the i-th second answer among the M second answers corresponding to the M questions in the second dialogue dataset, as well as the probability value corresponding to each sentiment label. M is an integer greater than 1, i takes the value of each integer from 1 to M, and N is an integer greater than 0. Based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first value, the target sentiment tag of the i-th second answer is determined, thereby determining the target sentiment tags of the M second answers; Add the corresponding target sentiment tag to the M second answers to obtain the first answer carrying the target sentiment tag for each of the M questions; The step of determining the target sentiment tag of the i-th second answer based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first value includes: If all N probability values ​​corresponding to the N sentiment tags of the i-th second answer are less than or equal to the first value, or if the N probability values ​​include a target probability value and the sentiment tag corresponding to the target probability value is without a sentiment tag, the i-th second answer carrying the j-th sentiment tag from the N sentiment tags is input into the multiple target sentiment models corresponding to the j-th sentiment tag to obtain multiple first judgment results for the j-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn; Based on the first determination rule, the second determination result of the j-th emotion tag is determined through multiple first determination results of the j-th emotion tag, so as to obtain N second determination results of the N emotion tags; Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the N second judgment results of the N sentiment labels.

2. The method according to claim 1, characterized in that, The step of obtaining the N sentiment labels of the i-th second answer among the M second answers corresponding to the M questions in the second dialogue dataset, and the probability value corresponding to each sentiment label, includes: Obtain M questions from the second dialogue dataset and M second answers corresponding to the M questions; The i-th second answer among the M second answers is input into a multi-label classification model to obtain N sentiment labels corresponding to the i-th second answer and the probability value corresponding to each sentiment label.

3. The method according to claim 1, characterized in that, The step of determining the target sentiment tag of the i-th second answer based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first value further includes: If there are P target probability values ​​among the N probability values, and not all of the P sentiment tags corresponding to the P target probability values ​​are without sentiment tags, the i-th second answer carrying the k-th sentiment tag among the P sentiment tags is input into the multiple target sentiment models corresponding to the k-th sentiment tag to obtain multiple first judgment results for the k-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, P is an integer greater than 0 and less than or equal to N, and k takes the value of each integer from 1 to P in sequence; Based on the first determination rule, the second determination result of the kth emotion tag is determined through multiple first determination results of the kth emotion tag, so as to obtain P second determination results of the P emotion tags; Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the P second judgment results of the P sentiment labels.

4. The method according to claim 1 or 3, characterized in that, The first determining rule includes: For multiple first judgment results of a sentiment label, those with the same first judgment result are grouped into one category for statistical analysis. Based on the first judgment result of the category with the most occurrences, the second judgment result of the sentiment label is determined.

5. The method according to claim 1 or 3, characterized in that, The second determining rule includes: For multiple second judgment results of multiple sentiment tags, if the multiple second judgment results are all the same, then the target sentiment tag is determined to be the sentiment tag corresponding to the second judgment result; otherwise, according to the priority order of sentiment tags, the target sentiment tag is determined to be the sentiment tag with the highest priority among the multiple second judgment results.

6. The method according to claim 1, characterized in that, The acquisition of the target personality includes: Obtain the prototype vector of each personality category from multiple personality categories, as well as the target person's historical dialogue data; The historical dialogue data is vectorized using a prototype network to obtain dialogue vector data. The prototype vector of each personality category is compared with the dialogue vector data to determine the personality category corresponding to the prototype vector with the highest similarity.

7. The method according to claim 1, characterized in that, The basic information of the person mentioned includes at least one of the following: identity information, personal experience information, values ​​information, and social relationship information.

8. A digital human generation device, characterized in that, include: The first acquisition module is used to acquire the first dialogue dataset, which includes M questions and the first answer carrying the target sentiment label for each question, where M is an integer greater than 1. The training module is used to train the large language model using the first dialogue dataset to obtain a first target large language model, which can output answers carrying emotions. The second acquisition module is used to acquire the target's personality type, language habits, and basic information about the person. The processing module is used to input the target personality category, the language habits and the basic information of the person into the first target large language model to obtain the second target large language model. The second target large language model can simulate a person's personality and language habits to output answers that carry emotions. The association module is used to associate the second target large language model with the digital human to obtain the target digital human corresponding to the second target large language model; The first acquisition module is specifically used for: Obtain the N sentiment labels of the i-th second answer among the M second answers corresponding to the M questions in the second dialogue dataset, as well as the probability value corresponding to each sentiment label. M is an integer greater than 1, i takes the value of each integer from 1 to M, and N is an integer greater than 0. Based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first value, the target sentiment tag of the i-th second answer is determined, thereby determining the target sentiment tags of the M second answers; Add the corresponding target sentiment tag to the M second answers to obtain the first answer carrying the target sentiment tag for each of the M questions; When the first acquisition module determines the target sentiment tag of the i-th second answer based on the relationship between the probability value corresponding to each sentiment tag of the i-th second answer and the first numerical value, it is specifically used for: If all N probability values ​​corresponding to the N sentiment tags of the i-th second answer are less than or equal to the first value, or if the N probability values ​​include a target probability value and the sentiment tag corresponding to the target probability value is without a sentiment tag, the i-th second answer carrying the j-th sentiment tag from the N sentiment tags is input into the multiple target sentiment models corresponding to the j-th sentiment tag to obtain multiple first judgment results for the j-th sentiment tag; wherein, the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn; Based on the first determination rule, the second determination result of the j-th emotion tag is determined through multiple first determination results of the j-th emotion tag, so as to obtain N second determination results of the N emotion tags; Based on the second determination rule, the target sentiment label of the i-th second answer is determined by the N second judgment results of the N sentiment labels.

9. An electronic device, characterized in that, Includes memory, transceiver, and processor: Memory, used to store computer programs; Transceiver, used to send and receive data under the control of the processor; A processor for reading a computer program from the memory and executing the digital human generation method as described in any one of claims 1 to 7.

10. A processor-readable storage medium, characterized in that, The processor-readable storage medium stores a computer program for causing the processor to perform the digital human generation method as described in any one of claims 1 to 7.

Images