Digital human generation method and device and electronic equipment

Abstract

The invention discloses a digital human generation method and device and electronic equipment, and belongs to the field of image processing. The method comprises the following steps: acquiring a first dialogue data set, wherein the first dialogue data set comprises M questions and a first answer which corresponds to each question and carries a target emotion label; training a large language model through the first dialogue data set to obtain a first target large language model which can output answers carrying emotions; obtaining a target character category, a language habit and character basic information; inputting the target character category, the language habit and the character basic information into the first target large language model to obtain a second target large language model, and the second target large language model can simulate the character and language habit of a person to output an answer carrying emotion; and associating the second target large language model with the digital person to obtain a target digital person corresponding to the second target large language model.

Description

Technical Field

[0001] The present application belongs to the field of artificial intelligence technology, and specifically relates to a method, device and electronic device for generating a digital human. Background Art

[0002] The current optimization of digital human technology generally focuses on improving the fidelity and naturalness of digital humans. The fidelity includes improving facial expressions and small movements, and the naturalness focuses on improving the fluency and similarity of speech. However, since digital humans have not yet mastered emotional understanding and expression, their effectiveness and anthropomorphism in practical applications are limited, resulting in a low level of intelligence in existing digital humans. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide a method, device and electronic device for generating a digital human, which can solve the problem that the digital human has not mastered emotional understanding and expression, resulting in a low level of intelligence.

[0004] In a first aspect, an embodiment of the present application provides a method for generating a digital human, comprising:

[0005] Obtain a first dialogue data set, where the first dialogue data set includes M questions and a first answer carrying a target sentiment label corresponding to each question, where M is an integer greater than 1;

[0006] Training the large language model using the first dialogue data set to obtain a first target large language model, wherein the first target large language model can output answers carrying emotions;

[0007] Obtain the target's personality type, language habits, and basic information about the person;

[0008] Inputting the target personality category, the language habits and the basic information of the person into the first target large language model to obtain a second target large language model, wherein the second target large language model can simulate the personality and language habits of the person and output an answer with emotion;

[0009] The second target large language model is associated with the digital human to obtain a target digital human corresponding to the second target large language model.

[0010] In a second aspect, an embodiment of the present application provides a digital human generation device, including:

[0011] A first acquisition module is used to acquire a first dialogue data set, where the first dialogue data set includes M questions and a first answer carrying a target emotion label corresponding to each question, where M is an integer greater than 1;

[0012] A training module, configured to train a large language model using the first dialogue data set to obtain a first target large language model, wherein the first target large language model can output an answer carrying emotion;

[0013] The second acquisition module is used to obtain the target personality category, language habits and basic information of the person;

[0014] A processing module, used for inputting the target personality category, the language habits and the basic information of the person into the first target large language model to obtain a second target large language model, wherein the second target large language model can simulate the personality and language habits of a person to output an answer with emotion;

[0015] The associating module is used to associate the second target large language model with the digital human to obtain a target digital human corresponding to the second target large language model.

[0016] In a third aspect, an embodiment of the present application further provides an electronic device, including a memory, a transceiver, and a processor:

[0017] A memory for storing a computer program; a transceiver for sending and receiving data under the control of a processor; and a processor for reading the computer program in the memory and executing the method as described in the first aspect above.

[0018] In a fourth aspect, an embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the method described in the first aspect above.

[0019] In an embodiment of the present application, by obtaining a first dialogue data set, the first dialogue data set includes M questions and a first answer with a target emotion label corresponding to each question, and the first dialogue data set is used to train the large language model to obtain a first target large language model, which can output an answer with emotion. Therefore, after the user inputs a question, the first target large language model can output an answer with emotion. In addition, by obtaining the target personality category, language habits and basic information of the character, the target personality category, the language habits and the basic information of the character are input into the first target large language model, and a second target large language model is obtained. The second target large language model can simulate the personality and language habits of the person and output an answer with emotion. The second target large language model is associated with a digital person to obtain a target digital person corresponding to the second target large language model. Therefore, after the user inputs the question, the second target large language model can simulate the personality and language habits of the person and output an answer with emotion, and the expression of the target digital person can make the conversation between the user and the target digital person more intelligent and realistic, and improve the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a flowchart of a method for generating a digital human provided in an embodiment of the present application;

[0021] Figure 2 Schematic diagram of input and output of the second target large language model provided in the embodiment of the present application;

[0022] Figure 3 It is a schematic diagram of the annotation of the emotion tags provided in the embodiment of the present application;

[0023] Figure 4 is a training diagram of a target emotion model provided in an embodiment of the present application;

[0024] Figure 5 is a schematic diagram of generating a first judgment result provided in an embodiment of the present application;

[0025] Figure 6 is a schematic diagram of the dialogue vector data and the prototype vector provided in the embodiment of the present application;

[0026] Figure 7 is a structural schematic diagram of a digital human generation device provided in an embodiment of the present application;

[0027] Figure 8 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0029] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way are interchangeable where appropriate, so that the embodiments of the present application can be implemented in an order other than those illustrated or described herein, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0030] The following describes the digital human generation method provided by the embodiment of the present application through specific embodiments and their application scenarios in conjunction with the accompanying drawings.

[0031] like Figure 1 As shown, the embodiment of the present application provides a method for generating a digital human, which may specifically include the following steps:

[0032] Step 101: Obtain a first dialogue data set, where the first dialogue data set includes M questions and a first answer carrying a target emotion label corresponding to each question, where M is an integer greater than 1.

[0033] Specifically, M questions and the first answer corresponding to each of the M questions are obtained, one question and the corresponding first answer are one dialogue data, M questions and M first answers are M dialogue data, and the M dialogue data constitute a first dialogue data set. Among them, each of the M first answers is an answer carrying a target emotion label.

[0034] Among them, the target emotion labels include but are not limited to: happy labels, angry labels, disappointed labels, fearful labels, surprised labels, no emotion labels, etc.

[0035] Step 102: Train a large language model using the first dialogue data set to obtain a first target large language model, where the first target large language model can output answers that carry emotions.

[0036] Specifically, the first dialogue data set is used as a sample data set to train the large language model, that is, each question in the first dialogue data set is input into the large language model, and the answer corresponding to each question is obtained based on the text understanding and reasoning capabilities of the large language model. The model parameters are continuously optimized through the answers obtained by the large language model and the first answers, so that the large language model reaches a convergence state and obtains the first target large language model. Inputting a user question into the first target large language model can obtain an answer with emotion.

[0037] Step 103, obtaining the target personality category, language habits and basic information of the person.

[0038] Furthermore, the basic character information includes but is not limited to at least one of the following: identity information, personal experience information, world outlook, values ​​and social relationship information.

[0039] Specifically, the target personality category, language habits, and basic character information are obtained, and these features are mapped into a personalized table of digital people, as shown in Table 1:

[0040] Table 1

[0041]

[0042] Among them, language habits may include but are not limited to: catchphrases (such as "You're amazing", "I'm going"), dialects (such as: "do"), common phrases (such as "Hehe", "Haha"), language style characteristics (such as: "formal language style characteristics"), etc. Basic personal information may include but are not limited to: identity information, personal experience information, three views information, social relationship information, etc. Among them, identity information may include: name, gender, age, interests, etc.; personal experience information may include: a brief introduction to life, important events experienced, etc.; three views information may include: world view, outlook on life, values, etc., and values may be dialectical materialist thoughts, including views and ideas on certain specific events.

[0043] Step 104: Input the target personality category, the language habit, and the basic personal information into the first target large language model to obtain a second target large language model, which can simulate a person's personality and language habit to output an answer with emotion.

[0044] Specifically, input the target personality category, language habit, and basic personal information into the first target large language model, and package the first target large language model through the target personality category, language habit, and basic personal information to obtain a second target large language model. This second large language model can simulate a person's personality, language habit, three views, etc., and generate an answer with emotion for the input question. For example: as Figure 2 shown, input the user question "Are you happy today?" into the second target large language model to obtain an answer with emotion "I'm very happy today

Happy

Happy

[0045] Step 105: Associate the second target large language model with a digital human to obtain a target digital human corresponding to the second target large language model.

[0046] Specifically, the second target large language model is associated with the digital human to obtain a target digital human corresponding to the second target large language model, and the target digital human can simulate the human's personality, language habits, three views, etc., that is, the target digital human is equivalent to a simulated human with personality, language habits, three views. If a voice question from the user is received, the voice question is converted into a text question through the voice-to-text technology and input into the second target large language model, and the second target large language model answers the question according to the personality, language habits, three views, etc., to obtain an emotional text answer, and the text answer is converted into a voice answer through the text-to-speech technology, and the voice answer is expressed by the target digital human, that is, the face driving module and the body driving module drive the target digital human to perform corresponding voice output, expression and action display according to the voice answer, so that the user can understand what the answer is, and the user can know the personality, language habits, three views, etc. of the target digital human through the answer expressed by the target digital human.

[0047] The following is a detailed description of the process of building a digital human:

[0048] First, it is necessary to determine the appearance, characteristics and personality of the digital human, including facial expressions, body proportions, clothing, etc. Then, the digital human is modeled and detailed by computer software. Modeling refers to creating the basic shape and structure of the digital human, while detailed carving is to add more details (such as skin texture, muscle lines, hair, clothes, etc.) to the basic model to make it more realistic. Adding bones and animation to the digital human is the key to making it active and expressive. Through bone binding and animation control technology, the digital human model can have movable joints and natural movements. Then, by adjusting the lighting settings and rendering parameters, the appearance of the digital human can be matched with the environment, and real shadows, reflections and refraction effects can be produced. Finally, post-production and special effects of the digital human are performed, including color correction, special effects, synthesis, etc., to further improve the visual quality and realism of the digital human. The second target large language model is bound and associated with the digital human to obtain a target digital human corresponding to the second target large language model. The target digital human is used to simulate a human to express the answer output by the second target large language model. In an embodiment of the present application, by obtaining a first dialogue data set, the first dialogue data set includes M questions and a first answer with a target emotion label corresponding to each question, and the first dialogue data set is used to train the large language model to obtain a first target large language model, which can output an answer with emotion. Therefore, after the user inputs a question, the first target large language model can output an answer with emotion. In addition, by obtaining the target personality category, language habits and basic information of the character, the target personality category, the language habits and the basic information of the character are input into the first target large language model, and a second target large language model is obtained. The second target large language model can simulate the personality and language habits of the person and output an answer with emotion. The second target large language model is associated with a digital person to obtain a target digital person corresponding to the second target large language model. Therefore, after the user inputs the question, the second target large language model can simulate the personality and language habits of the person and output an answer with emotion, and the expression of the target digital person can make the conversation between the user and the target digital person more intelligent and realistic, and improve the user experience.

[0049] As an optional specific embodiment, the step 101 of acquiring the first conversation data set specifically includes steps 1011 to 1014:

[0050] Step 1011, obtaining M questions in a second dialogue data set and M second answers corresponding to the M questions;

[0051] Step 1012, inputting the i-th second answer among the M second answers into a multi-label classification model to obtain N emotion labels corresponding to the i-th second answer and a probability value corresponding to each emotion label, where i is an integer from 1 to M in turn, and N is an integer greater than 0;

[0052] Step 1013, determining the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value, so as to determine the target emotion labels of the M second answers;

[0053] Step 1014: Add the corresponding target emotion label to the M second answers to obtain a first answer carrying the target emotion label corresponding to each of the M questions.

[0054] Specifically, M questions and the second answer corresponding to each of the M questions are obtained, one question and the corresponding second answer are one dialogue data, M questions and M second answers are M dialogue data, and the M dialogue data constitute a second dialogue data set.

[0055] The following is an explanation based on the i-th second answer among the M second answers:

[0056] The i-th second answer is input into the multi-label classification model to obtain the N emotion labels corresponding to the i-th second answer and the probability value corresponding to each emotion label, such as: {happy label: 0.5, sad label: 0.4, surprised label: 0.1, angry label: 0.3, fear label: 0.1, no emotion label: 0.1}. Since the N emotion labels and the probability value corresponding to each emotion label are obtained by the multi-label classification model, the sum of the N probability values ​​can be greater than 1.

[0057] Compare the probability value corresponding to each emotion label of the i-th second answer with the first numerical value (such as 0.5) to obtain the relationship between the probability value corresponding to each emotion label and the first numerical value. The target emotion label of the i-th second answer can be determined through the relationship between the probability value corresponding to each emotion label and the first numerical value.

[0058] It should be noted that the target emotion label of each of the M second answers is determined according to the process of the target emotion label of the i-th second answer, thereby determining the target emotion label of each of the M second answers.

[0059] Add the corresponding target emotion label to each of the M second answers, thereby obtaining the first answer carrying the target emotion label corresponding to each second answer. The M questions and the M first answers corresponding to the M questions constitute the first dialogue data set.

[0060] As an optional specific embodiment, the step 1013 determines the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value, specifically including:

[0061] In the case where the N probability values ​​corresponding to the N emotion labels of the ith second answer are all less than or equal to the first value, or in the case where there is a target probability value among the N probability values ​​and the emotion label corresponding to the target probability value is no emotion label, the ith second answer carrying the jth emotion label among the N emotion labels is respectively input into the multiple target emotion models corresponding to the jth emotion label to obtain multiple first judgment results of the jth emotion label; wherein the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn;

[0062] Based on the first determination rule, determining the second judgment result of the j-th emotion label through the multiple first judgment results of the j-th emotion label to obtain N second judgment results of the N emotion labels;

[0063] Based on the second determination rule, the target emotion label of the i-th second answer is determined through the N second judgment results of the N emotion labels.

[0064] Specifically, the probability value corresponding to each emotion label of the i-th second answer is compared with the first value. If the N probability values ​​corresponding to the N emotion labels of the i-th second answer are all less than or equal to the first value, or at least one of the N probability values ​​is greater than the target probability value of the first value, and the emotion labels corresponding to the target probability values ​​are all no emotion labels, then the target emotion label of the i-th second answer is determined according to the following rules:

[0065] The i-th second answer carrying different emotional labels is traversed through the target emotional model corresponding to the emotional label to obtain the first judgment result corresponding to each target emotional model, that is, the i-th second answer carrying the first emotional label is respectively input into the multiple target emotional models corresponding to the first emotional label to obtain the first judgment result output by the multiple target emotional models corresponding to the first emotional label; then the i-th second answer carrying the second emotional label is respectively input into the multiple target emotional models corresponding to the second emotional label to obtain the first judgment result output by the multiple target emotional models corresponding to the second emotional label; then the i-th second answer carrying the third emotional label is respectively input into the multiple target emotional models corresponding to the third emotional label to obtain the first judgment result output by the multiple target emotional models corresponding to the third emotional label; and so on, until the i-th second answer carrying the N-th emotional label is respectively input into the multiple target emotional models corresponding to the N-th emotional label to obtain the first judgment result output by the multiple target emotional models corresponding to the N-th emotional label.

[0066] For example: the first value is 0.5, N is 4, and the 4 emotion labels and the corresponding 4 probability values ​​are: {happy label: 0.4, sad label: 0.4, surprised label: 0.1, angry label: 0.3}. Since the 4 probability values ​​are all less than or equal to 0.5, the i-th second answer carrying the happy label is respectively input into multiple happy target emotion models (i.e., multiple target emotion models corresponding to the happy label), and the first judgment results output by multiple happy target emotion models are obtained; then the i-th second answer carrying the sad label is respectively input into multiple sad target emotion models (i.e., multiple target emotion models corresponding to the sad label), and the first judgment results output by multiple sad target emotion models are obtained; then the i-th second answer carrying the surprised label is respectively input into multiple surprised target emotion models (i.e., multiple target emotion models corresponding to the surprised label), and the first judgment results output by multiple surprised target emotion models are obtained; then the i-th second answer carrying the angry label is respectively input into multiple angry target emotion models (i.e., multiple target emotion models corresponding to the angry label), and the first judgment results output by multiple angry target emotion models are obtained.

[0067] It should be noted that each emotion label corresponds to multiple different target emotion models, that is, each emotion label corresponds to multiple (e.g., 3) different target emotion models, which are based on statistical feature classification models, word classification models, semantic classification models, etc. Multiple different target emotion models can judge the correctness of emotion labels from multiple different dimensions. Among them, the target emotion model is an emotion binary classification model.

[0068] The multiple first judgment results of the j-th emotion label are determined according to the first determination rule to determine the second judgment result of the j-th emotion label to obtain the second judgment result of each of the N emotion labels; the N second judgment results of the N emotion labels are determined according to the second determination rule to determine the target emotion label of the i-th second answer.

[0069] The first determination rule is described in detail below:

[0070] For multiple first judgment results of a sentiment tag, the first judgment results with the same value are counted as one category;

[0071] The second judgment result of the emotion tag is determined according to the first judgment result of the category with the largest number.

[0072] Specifically, if there are identical results among multiple first judgment results of the jth emotion tag, the identical first judgment results are classified into one category, and the number of first judgment results of the same category is counted. The number of first judgment results of different categories is compared, and the second judgment result of the jth emotion tag is determined according to the first judgment result of the category with the largest number.

[0073] For example, there are three first judgment results corresponding to the happy label, namely "is a happy label", "is not a happy label", and "is a happy label". The number of first judgment results of "is a happy label" is 2, and the number of first judgment results of "is not a happy label" is 1. Therefore, the second judgment result of the happy label needs to be determined as "is a happy label" based on the first judgment result of "is a happy label".

[0074] The second determination rule is described in detail below:

[0075] For multiple second judgment results of multiple emotion tags, if the multiple second judgment results are the same, the target emotion tag is determined to be the emotion tag corresponding to the second judgment result; otherwise, according to the priority order of the emotion tags, the target emotion tag is determined to be the emotion tag with the highest priority among the multiple second judgment results.

[0076] Specifically, for N second judgment results corresponding to N emotion tags, if the N second judgment results are all the same, the emotion tag corresponding to the second judgment result is the target emotion tag. If the N second judgment results are not all the same, then according to the priority order of the emotion tags, the emotion tag with the highest priority among the N emotion tags corresponding to the N second judgment results is determined as the target emotion tag.

[0077] For example, the second judgment result corresponding to the happy label is "is a happy label", the second judgment result corresponding to the angry label is "is not an angry label", and the second judgment result corresponding to the surprised label is "is a surprised label". Since the negative result of "is not an angry label" only indicates a non-angry label, and it is unknown what the emotional label is, the negative result of "is not an angry label" can be replaced with a non-emotional label, that is, the second judgment result corresponding to the happy label is "is a happy label", the second judgment result corresponding to the angry label is "is not an emotional label", and the second judgment result corresponding to the surprised label is "is a surprised label". Since the above three second judgment results are different, the surprised label is used as the target emotional label in the order of priority of the emotional labels (fear label>anger label>surprise label>sad label>happy label>no emotional label).

[0078] The training process of the target emotion model is described below through a specific embodiment:

[0079] like Figure 3 As shown, first, the user and the large language model have a conversation to obtain multiple rounds of conversation texts, and the conversation texts are annotated with emotion labels to obtain conversation texts with emotion labels, such as: happy label conversation texts, angry label conversation texts, and fear label conversation texts. This allows the conversation texts to contain emotional information (such as: happy, angry, fear, etc.), and only requires a small amount of annotation. Secondly, based on data enhancement capabilities, the existing small amount of annotated data is expanded. Then, based on the expanded data, multiple emotion binary classification models are trained (each emotion corresponds to multiple different emotion binary classification models) to obtain multiple trained target emotion models. For example: Figure 4 As shown, firstly, the data of the expanded happy label is obtained, and then multiple emotion binary classification models are trained based on the data of the expanded happy label to obtain the target emotion models corresponding to the three happy labels, namely, happy target emotion model 1, happy target emotion model 2, and happy target emotion model 3.

[0080] As an optional specific embodiment, the step 1013 determines the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value, and further includes:

[0081] In the case where there are P target probability values ​​among the N probability values, and not all of the P emotion labels corresponding to the P target probability values ​​are emotionless labels, the i-th second answer carrying the k-th emotion label among the P emotion labels is respectively input into the multiple target emotion models corresponding to the k-th emotion label to obtain multiple first judgment results of the k-th emotion label; wherein the target probability value is a probability value greater than the first value among the N probability values, P is an integer greater than 0 and less than or equal to N, and k is each integer from 1 to P in turn;

[0082] Based on the first determination rule, determining the second judgment result of the kth emotion label through the multiple first judgment results of the kth emotion label to obtain P second judgment results of the P emotion labels;

[0083] Based on the second determination rule, the target emotion label of the i-th second answer is determined through the P second judgment results of the P emotion labels.

[0084] Specifically, the probability value corresponding to each emotion label of the i-th second answer is compared with the first value. If P of the N probability values ​​corresponding to the N emotion labels of the i-th second answer are greater than the target probability values ​​of the first value, and the P emotion labels corresponding to the P target probability values ​​have a non-emotional label, then the target emotion label of the i-th second answer is determined according to the following rules:

[0085] The i-th second answer carrying different emotional labels among P emotional labels is traversed through the target emotional model corresponding to the emotional label to obtain the first judgment result corresponding to each target emotional model, that is, the i-th second answer carrying the first emotional label among the P emotional labels is respectively input into the multiple target emotional models corresponding to the first emotional label among the P emotional labels to obtain the first judgment result output by the multiple target emotional models corresponding to the first emotional label among the P emotional labels; then the i-th second answer carrying the second emotional label among the P emotional labels is respectively input into the multiple target emotional models corresponding to the second emotional label among the P emotional labels to obtain the first judgment result output by the multiple target emotional models corresponding to the second emotional label among the P emotional labels; and so on, until the i-th second answer carrying the P-th emotional label among the P emotional labels is respectively input into the multiple target emotional models corresponding to the P-th emotional label among the P emotional labels to obtain the first judgment result output by the multiple target emotional models corresponding to the P-th emotional label among the P emotional labels.

[0086] For example: Figure 5As shown, the first value is 0.5. First, the i-th second answer is obtained, and then the i-th second answer is input into the multi-label classification model to obtain the 4 emotion labels corresponding to the i-th second answer and the 4 probability values ​​corresponding to the 4 emotion labels. The 4 emotion labels and the corresponding 4 probability values ​​are: {happy label: 0.5, sad label: 0.4, surprised label: 0.1, angry label: 0.6}. Since the probability values ​​of the happy label and the angry label among the 4 probability values ​​are greater than 0.5, the target probability value corresponding to the happy label is 0.5, and the target probability value corresponding to the angry label is 0.6. The i-th second answer carrying the happy label is input into the happy target emotion model A1, the happy target emotion model A2, and the happy target emotion model A3 respectively, and the first judgment result corresponding to the happy target emotion model A1 is "happy label", the first judgment result corresponding to the happy target emotion model A2 is "happy label", and the first judgment result corresponding to the happy target emotion model A3 is "not happy label"; then the i-th second answer carrying the angry label is input into the angry target emotion model B1, the angry target emotion model B2, and the angry target emotion model B3 respectively, and the first judgment result corresponding to the angry target emotion model B1 is "angry label", the first judgment result corresponding to the angry target emotion model B2 is "angry label", and the first judgment result corresponding to the angry target emotion model B3 is "not angry label", thereby saving the calculation amount of emotion labels with lower probability values, saving resources, and improving efficiency.

[0087] The multiple first judgment results of the kth emotion label are determined according to the first determination rule to determine the second judgment result of the kth emotion label to obtain the second judgment result of each of the P emotion labels; the P second judgment results of the P emotion labels are determined according to the second determination rule to determine the target emotion label of the i-th second answer.

[0088] The first determination rule is described in detail below:

[0089] For multiple first judgment results of a sentiment tag, the first judgment results with the same value are counted as one category;

[0090] The second judgment result of the emotion tag is determined according to the first judgment result of the category with the largest number.

[0091] Specifically, if there are identical results among multiple first judgment results of the kth emotion tag, the identical first judgment results are classified into one category, and the number of first judgment results of the same category is counted. The number of first judgment results of different categories is compared, and the second judgment result of the kth emotion tag is determined according to the first judgment result of the category with the largest number.

[0092] For example, there are three first judgment results corresponding to the happy label, namely "is a happy label", "is not a happy label", and "is a happy label". The number of first judgment results of "is a happy label" is 2, and the number of first judgment results of "is not a happy label" is 1. Therefore, the second judgment result of the happy label needs to be determined as "is a happy label" based on the first judgment result of "is a happy label".

[0093] The second determination rule is described in detail below:

[0094] For multiple second judgment results of multiple emotion tags, if the multiple second judgment results are the same, the target emotion tag is determined to be the emotion tag corresponding to the second judgment result; otherwise, according to the priority order of the emotion tags, the target emotion tag is determined to be the emotion tag with the highest priority among the multiple second judgment results.

[0095] Specifically, for the P second judgment results corresponding to the P emotion tags, if the P second judgment results are all the same, the emotion tag corresponding to the second judgment result is the target emotion tag. If the P second judgment results are not all the same, then according to the priority order of the emotion tags, the emotion tag with the highest priority among the P emotion tags corresponding to the P second judgment results is determined as the target emotion tag.

[0096] For example, the second judgment result corresponding to the happy label is "is a happy label", the second judgment result corresponding to the angry label is "is not an angry label", and the second judgment result corresponding to the surprised label is "is a surprised label". Since the negative result of "is not an angry label" only indicates a non-angry label, and it is unknown what the emotional label is, the negative result of "is not an angry label" can be replaced with a non-emotional label, that is, the second judgment result corresponding to the happy label is "is a happy label", the second judgment result corresponding to the angry label is "is not an emotional label", and the second judgment result corresponding to the surprised label is "is a surprised label". Since the above three second judgment results are different, the surprised label is used as the target emotional label in the order of priority of the emotional labels (fear label>anger label>surprise label>sad label>happy label>no emotional label).

[0097] As an optional specific embodiment, the step 103 of obtaining the target personality specifically includes:

[0098] Obtaining the prototype vector of each personality category among multiple personality categories and the historical conversation data of the target person;

[0099] Vectorizing the historical conversation data through a prototype network to obtain conversation vector data;

[0100] The prototype vector of each personality category is compared with the dialogue vector data for similarity, and the personality category corresponding to the prototype vector with the highest similarity is determined as the target personality category.

[0101] Specifically, Figure 6 As shown, based on the Prototypical Networks (PN), the 16 personality types (such as scientist type, scholar type, commander type, etc.) in the Myers-Briggs Type Indicator (MBTI) are learned to obtain the prototype vectors of the corresponding personality categories, such as Figure 6 The prototype vectors of the scientist type, the prototype vectors of the scholar type, the prototype vectors of the commander type, etc. are shown. For the definition of a person's personality category, the historical dialogue data of a given character is vectorized by using the corresponding prototype network to obtain dialogue vector data, and the dialogue vector data is compared with the prototype vector of each personality category (i.e., similarity comparison), and the personality category corresponding to the prototype vector with the shortest distance (i.e., the highest similarity) is selected as the target personality category.

[0102] In summary, in the embodiment of the present application, the large language model is trained through M questions and the first answer carrying the target emotional label corresponding to each question, and the first target large language model capable of outputting answers carrying emotions is obtained. Therefore, after the user inputs the question, the first target large language model can output answers with emotions. In addition, by inputting the target personality category, language habits and basic information of the person into the first target large language model for packaging, a second target large language model capable of simulating the personality and language habits of the person to output answers carrying emotions is obtained, and the second target large language model is associated with the digital person to obtain the target digital person corresponding to the second target large language model. Therefore, after the user inputs the question, the second target large language model can simulate the personality, language habits, and world outlook of the person to output answers carrying emotions, and through the expression of the target digital person, the dialogue between the user and the target digital person can be made more intelligent and realistic, thereby improving the user experience.

[0103] The digital human generation method provided in the embodiment of the present application can be executed by a digital human generation device. In the embodiment of the present application, the digital human generation device executing the digital human generation method is taken as an example to illustrate the digital human generation device provided in the embodiment of the present application.

[0104] like Figure 7 As shown, the embodiment of the present application further provides a digital human generation device 700, which specifically includes:

[0105] A first acquisition module 701 is used to acquire a first dialogue data set, where the first dialogue data set includes M questions and a first answer carrying a target emotion label corresponding to each question, where M is an integer greater than 1;

[0106] A training module 702 is used to train a large language model using the first dialogue data set to obtain a first target large language model, where the first target large language model can output answers carrying emotions;

[0107] The second acquisition module 703 is used to acquire the target personality category, language habits and basic information of the person;

[0108] A processing module 704 is used to input the target personality category, the language habit and the basic information of the person into the first target large language model to obtain a second target large language model, wherein the second target large language model can simulate the personality and language habit of a person to output an answer with emotion;

[0109] The association module 705 is used to associate the second target large language model with the digital human to obtain a target digital human corresponding to the second target large language model.

[0110] As an optional specific embodiment, the first acquisition module 701 is specifically used to:

[0111] Obtaining M questions in a second dialogue data set and M second answers corresponding to the M questions;

[0112] Input the i-th second answer among the M second answers into the multi-label classification model to obtain N emotion labels corresponding to the i-th second answer and the probability value corresponding to each emotion label, where i is an integer from 1 to M in turn, and N is an integer greater than 0;

[0113] Determine the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value, so as to determine the target emotion labels of the M second answers;

[0114] The corresponding target emotion label is added to the M second answers to obtain a first answer carrying the target emotion label corresponding to each of the M questions.

[0115] As an optional specific embodiment, when the first acquisition module 701 determines the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first numerical value, it is specifically used to:

[0116] In the case where the N probability values ​​corresponding to the N emotion labels of the ith second answer are all less than or equal to the first value, or in the case where there is a target probability value among the N probability values ​​and the emotion label corresponding to the target probability value is no emotion label, the ith second answer carrying the jth emotion label among the N emotion labels is respectively input into the multiple target emotion models corresponding to the jth emotion label to obtain multiple first judgment results of the jth emotion label; wherein the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn;

[0117] Based on the first determination rule, determining the second judgment result of the j-th emotion label through the multiple first judgment results of the j-th emotion label to obtain N second judgment results of the N emotion labels;

[0118] Based on the second determination rule, the target emotion label of the i-th second answer is determined through the N second judgment results of the N emotion labels.

[0119] As an optional specific embodiment, when the first acquisition module 701 determines the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first numerical value, it is further specifically used to:

[0120] In the case where there are P target probability values ​​among the N probability values, and not all of the P emotion labels corresponding to the P target probability values ​​are emotionless labels, the i-th second answer carrying the k-th emotion label among the P emotion labels is respectively input into the multiple target emotion models corresponding to the k-th emotion label to obtain multiple first judgment results of the k-th emotion label; wherein the target probability value is a probability value greater than the first value among the N probability values, P is an integer greater than 0 and less than or equal to N, and k is each integer from 1 to P in turn;

[0121] Based on the first determination rule, determining the second judgment result of the kth emotion label through the multiple first judgment results of the kth emotion label to obtain P second judgment results of the P emotion labels;

[0122] Based on the second determination rule, the target emotion label of the i-th second answer is determined through the P second judgment results of the P emotion labels.

[0123] As an optional specific embodiment, the first determination rule includes:

[0124] For multiple first judgment results of a sentiment tag, the first judgment results with the same value are counted as one category;

[0125] The second judgment result of the emotion tag is determined according to the first judgment result of the category with the largest number.

[0126] As an optional specific embodiment, the second determination rule includes:

[0127] For multiple second judgment results of multiple emotion tags, if the multiple second judgment results are the same, the target emotion tag is determined to be the emotion tag corresponding to the second judgment result; otherwise, according to the priority order of the emotion tags, the target emotion tag is determined to be the emotion tag with the highest priority among the multiple second judgment results.

[0128] As an optional specific embodiment, when acquiring the target personality, the second acquisition module 703 is specifically used to:

[0129] Obtaining the prototype vector of each personality category among multiple personality categories and the historical conversation data of the target person;

[0130] Vectorizing the historical conversation data through a prototype network to obtain conversation vector data;

[0131] The prototype vector of each personality category is compared with the dialogue vector data for similarity, and the personality category corresponding to the prototype vector with the highest similarity is determined as the target personality category.

[0132] As an optional specific embodiment, the basic character information includes at least one of the following: identity information, personal experience information, world outlook, values ​​and social relationship information.

[0133] It should be noted here that the above-mentioned digital human generation device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned digital human generation method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as those in the method embodiment will not be described in detail here.

[0134] It should be noted that the division of units in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation. In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a processor-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program codes.

[0136] like Figure 8 As shown, an embodiment of the present application further provides an electronic device, including a memory 820, a transceiver 810, and a processor 800:

[0137] A memory 820, for storing computer programs;

[0138] A transceiver 810, configured to transmit and receive data under the control of the processor;

[0139] The processor 800 is used to read the computer program in the memory and execute the steps of the method for generating a digital human as described in any of the above embodiments.

[0140] Among them, Figure 8 In the embodiment, the bus architecture may include any number of interconnected buses and bridges, specifically linking together various circuits of one or more processors represented by processor 800 and memory represented by memory 820. The bus architecture may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are well known in the art and are therefore not further described herein. The bus interface provides an interface. The transceiver 810 may be a plurality of components, namely, a transmitter and a receiver, providing a unit for communicating with various other devices on a transmission medium, which transmission medium may include a wireless channel, a wired channel, an optical cable, and other transmission media. The processor 800 is responsible for managing the bus architecture and general processing, and the memory 820 may store data used by the processor 800 when performing operations.

[0141] The processor 800 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or a complex programmable logic device (CPLD). The processor may also adopt a multi-core architecture.

[0142] The processor calls the computer program stored in the memory to execute the digital human generation method provided in the embodiment of the present application according to the obtained executable instructions. The processor and the memory can also be arranged physically separately.

[0143] It should be noted here that the above-mentioned electronic device provided in the embodiment of the present application can implement all the method steps implemented in the above-mentioned digital human generation method embodiment, and can achieve the same technical effect. The parts and beneficial effects of this embodiment that are the same as the method embodiment will not be described in detail here.

[0144] An embodiment of the present application further provides a processor-readable storage medium, wherein the processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the above-mentioned digital human generation method.

[0145] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc.), optical storage (such as CD, DVD, BD, HVD, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD)), etc.

[0146] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0147] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer executable instructions. These computer executable instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0148] These processor executable instructions may also be stored in a processor readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the processor readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0149] These processor-executable instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0150] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for generating a digital human, It is characterized in that include: Obtain a first dialogue data set, where the first dialogue data set includes M questions and a first answer carrying a target sentiment label corresponding to each question, where M is an integer greater than 1; Training the large language model using the first dialogue data set to obtain a first target large language model, wherein the first target large language model can output answers carrying emotions; Obtain the target's personality type, language habits, and basic information about the person; Inputting the target personality category, the language habits and the basic information of the person into the first target large language model to obtain a second target large language model, wherein the second target large language model can simulate the personality and language habits of the person and output an answer with emotion; The second target large language model is associated with the digital human to obtain a target digital human corresponding to the second target large language model.

2. The method according to claim 1, It is characterized in that The obtaining of the first dialogue data set includes: Obtaining M questions in a second dialogue data set and M second answers corresponding to the M questions; Input the i-th second answer among the M second answers into the multi-label classification model to obtain N emotion labels corresponding to the i-th second answer and the probability value corresponding to each emotion label, where i is an integer from 1 to M in turn, and N is an integer greater than 0; Determine the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value, so as to determine the target emotion labels of the M second answers; The corresponding target emotion label is added to the M second answers to obtain a first answer carrying the target emotion label corresponding to each of the M questions.

3. The method according to claim 2, It is characterized in that The step of determining the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value includes: In the case where the N probability values ​​corresponding to the N emotion labels of the ith second answer are all less than or equal to the first value, or in the case where there is a target probability value among the N probability values ​​and the emotion label corresponding to the target probability value is no emotion label, the ith second answer carrying the jth emotion label among the N emotion labels is respectively input into the multiple target emotion models corresponding to the jth emotion label to obtain multiple first judgment results of the jth emotion label; wherein the target probability value is the probability value among the N probability values ​​that is greater than the first value, and j takes the value of each integer from 1 to N in turn; Based on the first determination rule, determining the second judgment result of the j-th emotion label through the multiple first judgment results of the j-th emotion label to obtain N second judgment results of the N emotion labels; Based on the second determination rule, the target emotion label of the i-th second answer is determined through the N second judgment results of the N emotion labels.

4. The method according to claim 3, It is characterized in that The step of determining the target emotion label of the i-th second answer according to the relationship between the probability value corresponding to each emotion label of the i-th second answer and the first value further includes: In the case where there are P target probability values ​​among the N probability values, and not all of the P emotion labels corresponding to the P target probability values ​​are emotionless labels, the i-th second answer carrying the k-th emotion label among the P emotion labels is respectively input into the multiple target emotion models corresponding to the k-th emotion label to obtain multiple first judgment results of the k-th emotion label; wherein the target probability value is a probability value greater than the first value among the N probability values, P is an integer greater than 0 and less than or equal to N, and k is each integer from 1 to P in turn; Based on the first determination rule, determining the second judgment result of the kth emotion label through the multiple first judgment results of the kth emotion label to obtain P second judgment results of the P emotion labels; Based on the second determination rule, the target emotion label of the i-th second answer is determined through the P second judgment results of the P emotion labels.

5. The method according to claim 3 or 4, It is characterized in that The first determination rule includes: For multiple first judgment results of a sentiment tag, the first judgment results with the same value are counted as one category; The second judgment result of the emotion tag is determined according to the first judgment result of the category with the largest number.

6. The method according to claim 3 or 4, It is characterized in that The second determination rule includes: For multiple second judgment results of multiple emotion tags, if the multiple second judgment results are the same, the target emotion tag is determined to be the emotion tag corresponding to the second judgment result; otherwise, according to the priority order of the emotion tags, the target emotion tag is determined to be the emotion tag with the highest priority among the multiple second judgment results.

7. The method according to claim 1, It is characterized in that The obtaining of the target personality comprises: Obtaining the prototype vector of each personality category among multiple personality categories and the historical conversation data of the target person; Vectorizing the historical conversation data through a prototype network to obtain conversation vector data; The prototype vector of each personality category is compared with the dialogue vector data for similarity, and the personality category corresponding to the prototype vector with the highest similarity is determined as the target personality category.

8. The method according to claim 1, It is characterized in that The basic character information includes at least one of the following: identity information, personal experience information, world outlook, values ​​and social relationship information.

9. A digital human generation device, It is characterized in that include: A first acquisition module is used to acquire a first dialogue data set, where the first dialogue data set includes M questions and a first answer carrying a target emotion label corresponding to each question, where M is an integer greater than 1; A training module, configured to train a large language model using the first dialogue data set to obtain a first target large language model, wherein the first target large language model can output an answer carrying emotion; The second acquisition module is used to obtain the target personality category, language habits and basic information of the person; A processing module, used for inputting the target personality category, the language habits and the basic information of the person into the first target large language model to obtain a second target large language model, wherein the second target large language model can simulate the personality and language habits of a person to output an answer with emotion; The associating module is used to associate the second target large language model with the digital human to obtain a target digital human corresponding to the second target large language model.

10. An electronic device, It is characterized in that Including memory, transceiver, processor: Memory for storing computer programs; a transceiver, for transmitting and receiving data under the control of the processor; A processor, configured to read the computer program in the memory and execute the method for generating a digital human as claimed in any one of claims 1 to 8.

11. A processor-readable storage medium, It is characterized in that The processor-readable storage medium stores a computer program, and the computer program is used to enable the processor to execute the digital human generation method according to any one of claims 1 to 8.

Images