Intelligent dialogue method, device, electronic device and storage medium for digital human

Abstract

The present disclosure provides a method, device, electronic device and storage medium for intelligent dialogue of digital human beings, which relates to technical fields such as digital human beings, virtual technology and artificial intelligence. The specific implementation scheme is: obtaining the character description information of the digital human being; obtaining the corresponding professional knowledge base based on the character description information; using the pre-trained digital human dialogue system, based on the character description information of the digital human being and the professional knowledge base, generating dialogue response information corresponding to the dialogue question information. The technology disclosed in the present disclosure can give the digital human a character and realize a very professional and very accurate dialogue response capability.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, specifically to the technical fields of digital human, virtual technology, artificial intelligence, and more particularly to an intelligent dialogue method, device, electronic device, and storage medium for digital human. Background Art

[0002] Virtual digital humans that integrate artificial intelligence (AI) technologies in various fields such as voice and text have experienced explosive growth in recent years. Currently, digital humans on the market mainly use traditional dialogue systems to communicate with users.

[0003] During operation, a traditional dialogue system can obtain the user's speech content through voice recognition; then obtain the reply text from the dialogue system based on the user's speech content; and convert the reply text into voice to reply to the user. Summary of the invention

[0004] The present invention provides a method, device, electronic device and storage medium for intelligent dialogue of a digital human.

[0005] According to one aspect of the present disclosure, a method for intelligent dialogue of a digital human is provided, comprising:

[0006] Obtain the character description information of the digital person;

[0007] Based on the persona description information, obtain a corresponding professional knowledge base;

[0008] A pre-trained digital human dialogue system is used to generate dialogue response information corresponding to dialogue question information based on the digital human's personality description information and the professional knowledge base.

[0009] According to another aspect of the present disclosure, there is provided an intelligent dialogue device for a digital human, comprising:

[0010] A character acquisition module is used to obtain character description information of a digital human;

[0011] A knowledge base acquisition module, used to acquire a corresponding professional knowledge base based on the persona description information;

[0012] The generation module is used to use a pre-trained digital human dialogue system to generate dialogue response information corresponding to the dialogue question information based on the digital human's personality description information and the professional knowledge base.

[0013] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any possible implementation manner and the aspects described above.

[0017] According to yet another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method of the above-mentioned aspect and any possible implementation manner.

[0018] According to yet another aspect of the present disclosure, a computer program product is provided, including a computer program, wherein when the computer program is executed by a processor, the computer program implements the above-mentioned aspects and any possible implementation method.

[0019] According to the technology disclosed in the present invention, it is possible to give a digital human a personality and achieve a very professional and very accurate dialogue response capability.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0022] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0023] Figure 2 is a schematic diagram according to a second embodiment of the present disclosure;

[0024] Figure 3 is a schematic diagram according to a third embodiment of the present disclosure;

[0025] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0026] Figure 5 The block diagram is a block diagram of an electronic device for implementing the method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] Obviously, the described embodiments are only part of the embodiments of the present disclosure, but not all of them. Based on the embodiments of the present disclosure, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure.

[0029] It should be noted that the terminal devices involved in the embodiments of the present disclosure may include but are not limited to mobile phones, personal digital assistants (PDAs), wireless handheld devices, tablet computers and other smart devices; display devices may include but are not limited to personal computers, televisions and other devices with display functions.

[0030] In addition, the term "and / or" in this article is only a description of the association relationship between the associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0031] The "intelligent capabilities" of the traditional dialogue systems used by existing digital humans are very limited. They can only provide users with some simple responses as a third party, and the accuracy and naturalness of the dialogue responses are poor.

[0032] With the new round of AI craze, various large language models (LLMs) have shown the dawn of artificial general intelligence (AGI) with their unexpected "emergence" capabilities. The dialogue system based on large speech models will completely change the experience of human-computer interaction and be applied to all walks of life. Based on this, the present disclosure can consider using LLMs to replace the dialogue system of digital humans, effectively improving the intelligent ability of digital humans, so that digital humans can output more accurate, more natural, and more expected response information in the dialogue.

[0033] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure; Figure 1As shown, this embodiment provides a digital human intelligent dialogue method, which is applied to a digital human intelligent dialogue device. The digital human intelligent dialogue device is applied in a digital human system, and specifically may include the following steps:

[0034] S101, obtaining the character description information of the digital human;

[0035] S102, based on the persona description information, obtaining a corresponding professional knowledge base;

[0036] S103: Using the pre-trained digital human dialogue system, based on the digital human's personality description information and the professional knowledge base, generate dialogue response information corresponding to the dialogue question information.

[0037] In this embodiment, a specific personality setting can be assigned to the digital human. Specifically, the personality setting description information of the digital human can be input into the intelligent dialogue device of the digital human to assign the personality setting corresponding to the personality setting description information to the digital human.

[0038] In order to provide more professional services, in this embodiment, the corresponding professional knowledge base can also be obtained based on the character description information to provide professional knowledge support for the digital human's dialogue system when responding to the dialogue.

[0039] The digital human dialogue system of this embodiment can be implemented based on pre-trained LLMs, such as the pre-trained Generative Pre-trained Transformer (GPT) 3.5 model and GPT4 model developed by OpenAI, and the Large Language Model Meta AI (LLaMa) developed by Meta AI, etc.

[0040] The digital human dialogue system of this embodiment can adopt LLMs, and based on the digital human's personality description information and professional knowledge base, with the personality assigned to the digital human and reference to the professional knowledge base, very professionally generate dialogue response information corresponding to the dialogue question information.

[0041] The intelligent dialogue method for digital humans in this embodiment adopts the dialogue system of digital humans, and can generate highly professional and accurate dialogue response information for dialogue question information based on the personality description information of the digital humans and the corresponding professional knowledge base, thereby endowing the digital humans with highly professional and accurate dialogue response capabilities.

[0042] Figure 2 is a schematic diagram of a second embodiment of the present disclosure; the intelligent dialogue method of a digital human in this embodiment, in the above Figure 1 Based on the technical solutions of the embodiments shown in the figure, the technical solutions of the present disclosure are further described in more detail. Figure 2As shown, the intelligent dialogue method of the digital human in this embodiment may specifically include the following steps:

[0043] S201, obtaining the character description information of the digital human; the character description information includes character role information, character information of the character, description information of the tags carried in the dialogue response of the character, and example samples of the dialogue response of the character;

[0044] The persona role information of this embodiment may be customer service of a certain industry, or customer service of a certain field of the industry, and may have stronger professionalism. For example, the persona role information may be customer service of the credit field of a bank.

[0045] The character information of the human role in this embodiment can be used to identify the style of the human role in generating dialogue replies. For example, the character information of the human role can be: "You are a dignified, steady but friendly customer service", or "Your personality is lively and smart", or "You are now a customer service staff of xx Bank, please play this role to provide services", etc.

[0046] The character description information of this embodiment also includes: description information of the label carried in the dialogue response of the character, wherein the label may include an emotion label; or further, the label may also include a body control label. In the scenario of this embodiment, the dialogue response of the character needs to carry the label synchronously. At the same time, the control scenario of the label needs to be limited in the description information. For example, the emotion label can be used to control the emotion change during speech synthesis; it can also be used to control the facial expression change of the digital person during facial expression synthesis. If the digital person only provides dialogue responses in voice mode, the label can only include the emotion label. If the digital person uses a 3D CG image such as a 3D high-precision realistic portrait, a 3D beauty-style portrait, a 3D cartoon-style portrait or a 3D cartoon animal image to make a dialogue response, the label can further include a body control label, which can be used to control the body movements of the digital person such as gestures when responding to the dialogue.

[0047] Optionally, in this embodiment, an emotion tag library and a body control tag library may be pre-created, and the emotion tag library may include multiple emotion tags, such as happy, natural, sad, etc., and the style of each emotion tag when controlling speech synthesis may be defined in the emotion tag library. In addition, the description information of the facial expression control corresponding to each emotion tag may be defined in the emotion tag library. For example, when happy, the eye shape may be controlled to be a smiling eye shape, the mouth shape may be controlled to be a happy state with the corners of the mouth raised, etc. When sad, the mouth shape may be controlled to be a sad state with the corners of the mouth lowered, etc. In practical applications, the facial expression movements of the digital human may be controlled according to the state of the facial expression corresponding to each emotion tag configured by the professional designer of the digital human.

[0048] The limb control tag library may include multiple limb motion tags of the digital human, such as greeting, stopping, making phone calls, nodding, shaking heads, etc. The descriptive information of the control of the limb motion corresponding to each limb control tag may also be limited in the limb control tag library. For example, when greeting, the digital human may be controlled to wave his hands. For example, when the digital human makes a phone call, the digital human's right fist may be controlled to move to the right ear. For another example, when the digital human nods, the digital human's head may be controlled to point downward three times in a row, etc. In actual applications, the limb motion corresponding to each limb control tag may be configured by the professional designer of the digital human, and the digital human may be controlled to complete the corresponding limb motion.

[0049] The character description information of this embodiment also includes: example samples of dialogue responses of the character. The number of example samples of dialogue responses of the character can be set according to needs. For example, the number of example samples corresponding to each type of label can include more than dozens of samples, so that the digital human dialogue system can learn to carry the corresponding label in the dialogue response based on the example samples corresponding to each type of label.

[0050] Based on the above, it can be known that the character description information of this embodiment not only includes character information, but also includes personality information of the character, label description information in the corresponding dialogue response, and example samples of the dialogue response of the character. The character description information contains very rich and comprehensive information, so that the digital human dialogue system can fully grasp the style of the character's dialogue response based on the above character description information, and learn the ability to provide dialogue responses with the character.

[0051] S202, based on the character role information in the character description information, obtaining a professional knowledge base of the character role from a plurality of pre-created professional knowledge bases of known character roles;

[0052] In this embodiment, each person character information corresponds to a field. Before the implementation of this embodiment, a professional knowledge base of known person characters corresponding to various fields can be pre-created. The professional knowledge base of the person character includes professional knowledge in the field where the person character is located, which can provide very professional knowledge support for the digital human dialogue system to respond to dialogues.

[0053] For example, before step S202, the following steps may be specifically included:

[0054] (1) For each of the multiple known human roles, obtain professional knowledge in the field to which the known human role belongs;

[0055] In this embodiment, the professional knowledge in the field to which the known human role belongs can be obtained from the information base of the field to which the known human role belongs. Alternatively, the corresponding professional knowledge can be configured by professional staff in the field to which the known human role belongs. Alternatively, other methods can be used to obtain the professional knowledge in the field to which the known human role belongs from other channels, which is not limited here.

[0056] (2) Segment the professional knowledge according to the preset text length to obtain multiple knowledge segments;

[0057] In this embodiment, the acquired professional knowledge in the field to which the known character role belongs may be in the form of a long text, which may include multiple sentences, or multiple paragraphs, etc.

[0058] Specifically, all text contents in the professional knowledge can be spliced ​​together, and then divided into pieces according to a preset text length to obtain multiple knowledge fragments. The preset text length can be set according to actual needs, such as 500 words, 1000 words, etc., or any length, which is not limited here.

[0059] (3) Extract the characteristic expressions of each knowledge fragment;

[0060] Specifically, a pre-trained feature extraction model may be used to extract the feature expression of each knowledge fragment, and the feature expression of the knowledge fragment may be in the form of a vector.

[0061] (4) Based on each knowledge fragment and its characteristic expression, a professional knowledge base of corresponding known human roles is constructed.

[0062] Specifically, each knowledge fragment and the corresponding feature expression of the field to which the known character belongs are stored together in the professional knowledge base of the known character. In other words, the professional knowledge base of the known character also includes the corresponding relationship between each knowledge fragment and the corresponding feature expression.

[0063] The above steps (1)-(4) are pre-established offline before the digital human dialogue system provides dialogue services using human role information.

[0064] Through the above method, a professional knowledge base of known human roles in various fields can be created in advance, so that when it is necessary to use known human roles in the corresponding field to provide dialogue services, the corresponding professional knowledge base can be obtained in a timely and accurate manner.

[0065] For the character description information of the same digital person, before providing services with the corresponding character role, steps S201 and S202 are only performed once.

[0066] S203, obtaining the dialogue question information and historical dialogue information of the digital human;

[0067] In this embodiment, the dialogue question information may be dialogue question information in text form obtained through voice collection and voice recognition. The historical dialogue information may include historical multi-round dialogue information, that is, dialogue question information of users and dialogue response information of digital humans in historical multi-round dialogues. The historical dialogue information may also be saved in the form of text during the dialogue process, and can be directly obtained when used.

[0068] It should be noted that if the user's conversation question information is the first question information of a round of conversation, the historical conversation information is empty.

[0069] S204, using the digital human dialogue system, based on the human role information, the personality information of the human role, the description information of the label carried in the dialogue response of the human role, the sample of the dialogue response of the human role and the professional knowledge base of the human role, and referring to the historical dialogue information, generate dialogue response information for the dialogue question information, wherein the dialogue response information carries the label;

[0070] In this embodiment, the tag carried in the dialogue reply information is used to limit the text in the dialogue reply. For example, the tag can be specifically located before the corresponding text.

[0071] In this embodiment, the historical dialogue information may include historical multi-round dialogue information, that is, dialogue question information of users and dialogue response information of digital humans in historical multi-round dialogues.

[0072] By adopting the method of step S204 above, a digital human dialogue system can be used, and by fully and comprehensively referring to the human role information, the personality information of the human role, the descriptive information of the tags carried in the human role's dialogue replies, example samples of the human role's dialogue replies, the human role's professional knowledge base, and historical dialogue information, dialogue response information for dialogue question information can be accurately and efficiently generated.

[0073] It should be noted that if the user's dialogue question information is the first question information of a round of dialogue, the historical dialogue information is empty at this time. That is, at this time, the digital human dialogue system can be used to generate dialogue response information for the dialogue question information based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, the sample samples of the dialogue response of the human role, and the professional knowledge base of the human role.

[0074] In specific implementation, step S204 may include the following two implementation methods:

[0075] The first implementation method may include the following steps:

[0076] (1) Based on the character information, personality information, description information of the tags carried in the dialogue responses of the character, and example samples of the dialogue responses of the character, control the ability of the digital human dialogue system to learn the dialogue response information of the character;

[0077] (2) The learned digital human dialogue system is used to generate dialogue response information for replying to the dialogue question information based on dialogue question information, historical dialogue information, and the professional knowledge base of the human character, and the dialogue response information carries a label.

[0078] For example, the dialogue system of the digital human after learning can be used to obtain relevant knowledge fragments from the professional knowledge base of the human character based on the dialogue question information and the historical dialogue information; for example, when this step is specifically implemented, the corresponding feature expressions of the dialogue question information and the historical dialogue information can be obtained respectively; wherein if the historical dialogue information includes multiple rounds, the feature expressions of each question information and each reply information can be obtained respectively. The feature expression of this embodiment can be in the form of a vector. When extracting the feature expression, a pre-trained feature extraction model can be used to implement it. Then, based on the corresponding feature expressions of the dialogue question information and the historical dialogue information, the feature expression of the relevant knowledge fragment is obtained from the professional database of the human character by vector search. Then, based on the correspondence between the feature expression of the knowledge fragment in the professional database of the human character and the knowledge fragment, the relevant knowledge fragment is obtained. In this embodiment, the relevant knowledge fragment includes the relevant knowledge fragment of the dialogue question information and the relevant knowledge fragment of the historical dialogue information. Then, the dialogue system of the digital human after learning is used to generate dialogue reply information for replying to the dialogue question information based on the dialogue question information, the historical dialogue information, and the relevant knowledge fragment.

[0079] In this embodiment, relevant knowledge fragments are obtained by adopting a vector search method. Since the vector search method has higher accuracy, the accuracy of the obtained relevant knowledge fragments can be effectively improved, and thus the accuracy of the dialogue response information generated by the digital human dialogue system can be effectively improved.

[0080] In the first implementation, the dialogue system of the digital person may not be upgraded, that is, the parameters of the dialogue system of the digital person may not be changed. However, the dialogue system of the digital person can still be controlled, and the ability of the dialogue response information of the human character can be fully learned based on the human character information, the personality information of the human character, the description information of the tags carried in the dialogue response of the human character, and the sample samples of the dialogue response of the human character; and further, the dialogue response information of the dialogue question information, the historical dialogue information and the professional knowledge base of the human character can be referred to accurately and efficiently.

[0081] The second implementation method may specifically include the following steps:

[0082] (a) Based on the character information, personality information, description information of the tags carried in the dialogue responses of the character, and example samples of the dialogue responses of the character, the dialogue system of the digital human is learned to obtain an upgraded dialogue system of the digital human;

[0083] (b) An upgraded dialogue system based on a digital human generates dialogue response information for replying to dialogue question information based on dialogue question information, historical dialogue information, and the professional knowledge base of the human character.

[0084] Compared with the first implementation, in the second implementation, the digital human dialogue system can be learned based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, and the sample of the dialogue response of the human role, so as to obtain the upgraded dialogue system of the digital human. The learning process can be regarded as a process of fine-tuning the dialogue system of the digital human, and the parameters of the dialogue system of the digital human have changed.

[0085] For example, the digital human's upgraded dialogue system can also obtain relevant knowledge fragments from the professional knowledge base of the human character based on dialogue question information and historical dialogue information; and then use the digital human's upgraded dialogue system to generate dialogue response information based on dialogue question information, historical dialogue information, and relevant knowledge fragments to respond to the dialogue question information.

[0086] It should be noted that in this implementation, for each character description information, step (a) is only executed once, and then the upgraded dialogue system of the digital person is saved. Then, the subsequent dialogues of the character role are based on the upgraded dialogue system of the digital person to provide dialogue services, and there is no need to learn the dialogue response ability of the character role in each round of dialogue, which can further improve the ability of the digital person to provide dialogue services with the character role. During the dialogue process, for each sentence of dialogue question information, step (b) can be used to professionally and intelligently generate dialogue response information to reply to the dialogue question information.

[0087] S205, parsing the dialogue reply information to obtain the carried tags;

[0088] For example, a pre-trained parsing module may be used to parse the conversation reply information and obtain the labels carried therein.

[0089] S206: if the tag carried in the dialogue reply information includes an emotional tag, synthesizing the reply voice of the digital human based on the emotional tag carried in the dialogue reply information and the dialogue reply information;

[0090] S207, controlling the facial expression of the digital human based on the emotion tag carried in the dialogue reply information;

[0091] S208: If the tag carried in the dialogue reply information includes a body control tag, based on the body control tag carried in the dialogue reply information, perform body movement control on the digital human in the process of replying the voice.

[0092] For example, in this embodiment, based on the pre-created emotional tag library and body control tag library, it is possible to detect whether each tag in the dialogue reply information is an emotional tag or a body control tag. For each emotional tag, based on the emotional tag and dialogue reply information carried in the dialogue reply information, the corresponding reply voice of the digital human is generated.

[0093] It should be noted that, in this embodiment, the emotion tag library can also be referenced to determine whether each emotion tag is used to limit the control of facial expressions. If a certain emotion tag can control the facial expression of the digital human, step S207 needs to be adopted to control the facial expression of the digital human.

[0094] For the parsed limb control tags, based on the description information of the limb control in the limb control tags, the limb movements of the digital human are controlled in the process of replying to the voice.

[0095] In this embodiment, during the conversation between the digital humans, the reply of each round of conversation can be realized according to the method of steps S203-S208.

[0096] For example, for a digital person with a cartoon image, the generated dialogue reply message can be: <happy> Hello, <natural> Ms. Wang [hello], <natural> Do you have anything else to ask me?

[0097] The parsing module extracts emotional tags such as "happy" and "natural" and the body control tag "greeting" from the dialogue reply information. Then, the corresponding reply voice of the digital human can be synthesized based on each emotional state. At the same time, the digital human can also be controlled to automatically make corresponding emotional expressions and body movements when synthesizing the picture, making the digital human's service more intelligent, more professional, and more expressive.

[0098] Based on the above-obtained emotional control and body movement control of the digital human, the display screen of the digital human can be rendered; and combined with the digital human's reply voice, the digital human's audio and video can be output and displayed to the user.

[0099] The dialogue system of the digital human used in this embodiment is a dialogue system based on LLMs, which has very professional dialogue response capabilities. In this embodiment, on the basis of the dialogue system based on LLMs, a human role is given to the digital human, the dialogue interaction capability of the digital human is enhanced, and a more intelligent dialogue response is achieved. In the process of interaction between the digital human and the outside world, its interactive expressiveness is enriched. For example, intelligent emotional expression and intelligent action expression can be achieved, so that the digital human can conduct dialogue and question and answer like a real person, and at the same time, make some corresponding body movements smoothly and naturally, so as to achieve the effect of the digital human being being indistinguishable from the real person.

[0100] The intelligent dialogue method of the digital human of this embodiment, by introducing LLMs as the dialogue system of the digital human, endows the digital human with the ability of a professional human role based on the character description information of the digital human and the corresponding professional knowledge base, so that the digital human can provide more professional and more accurate dialogue responses based on the professional human role. Moreover, by adopting the technical solution of this embodiment, the digital human also has the ability of emotional expression and action expression, so that the digital human can simultaneously realize the integrated task of action expression, emotional expression and dialogue response, which can greatly improve the professionalism and accuracy of the digital human's interactive response, and at the same time improve the digital human's emotional and action expression. The solution of this embodiment can be applied to products related to virtual digital humans, and is widely used and very practical.

[0101] Figure 3 is a schematic diagram according to the third embodiment of the present disclosure; Figure 3 As shown, this embodiment provides an intelligent dialogue device 300 of a digital human, including:

[0102] A character setting acquisition module 301 is used to acquire character setting description information of a digital human;

[0103] A knowledge base acquisition module 302 is used to acquire a corresponding professional knowledge base based on the persona description information;

[0104] The generation module 303 is used to use the pre-trained digital human dialogue system to generate dialogue response information corresponding to the dialogue question information based on the digital human's personality description information and the professional knowledge base.

[0105] The intelligent dialogue device 300 of the digital human of this embodiment can realize the implementation principle and technical effect of the intelligent dialogue of the digital human by adopting the above modules, which is the same as the implementation of the above related method embodiments. For details, please refer to the records of the above related method embodiments, which will not be repeated here.

[0106] Figure 4 is a schematic diagram according to a fourth embodiment of the present disclosure; Figure 4As shown, this embodiment provides a digital human intelligent dialogue device 400, including the above Figure 3 The modules of the digital human intelligent dialogue device 300 with the same name and function are: a human setting acquisition module 401 , a knowledge base acquisition module 402 and a generation module 403 .

[0107] In one embodiment of the present disclosure, the character description information includes character role information, personality information of the character, description information of tags carried in the dialogue response of the character, and example samples of the dialogue response of the character; the tags include emotion tags;

[0108] Furthermore, the tag also includes a limb control tag.

[0109] Further optionally, in one embodiment of the present disclosure, the knowledge base acquisition module 402 is used to:

[0110] Based on the human role information in the human description information, a professional knowledge base of the human role is obtained from a plurality of pre-created professional knowledge bases of known human roles.

[0111] Further optionally, in one embodiment of the present disclosure, the intelligent dialogue device 400 of the digital human further includes:

[0112] The knowledge acquisition module 404 is used to acquire, for each known human role among a plurality of known human roles, the professional knowledge of the field to which the known human role belongs;

[0113] A segmentation module 405 is used to segment the professional knowledge according to a preset text length to obtain multiple knowledge segments;

[0114] An extraction module 406 is used to extract the characteristic expression of each of the knowledge fragments;

[0115] The construction module 407 is used to construct a professional knowledge base of the corresponding known human role based on each of the knowledge fragments and the characteristic expressions of each of the knowledge fragments.

[0116] Further optionally, in one embodiment of the present disclosure, the generating module 403 is used to:

[0117] The digital human dialogue system is adopted to generate the dialogue response information for the dialogue question information based on the human role information, the personality information of the human role, the description information of the label carried in the dialogue response of the human role, the example samples of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, wherein the dialogue response information carries the label.

[0118] Further optionally, in one embodiment of the present disclosure, the generating module 403 is used to:

[0119] Based on the human character information, the personality information of the human character, the description information of the tags carried in the dialogue response of the human character, and the example samples of the dialogue response of the human character, the ability of the dialogue system of the digital human to learn the dialogue response information of the human character is controlled;

[0120] The dialogue system of the digital human after learning is used to generate the dialogue response information for responding to the dialogue question information based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human role.

[0121] Further optionally, in one embodiment of the present disclosure, the generating module 403 is used to:

[0122] Using the learned dialogue system of the digital human, based on the dialogue question information and the historical dialogue information, to obtain relevant knowledge fragments from the professional knowledge base of the human character;

[0123] The dialogue system of the digital human after learning is used to generate the dialogue reply information for replying to the dialogue question information based on the dialogue question information, the historical dialogue information, and the relevant knowledge fragments.

[0124] Further optionally, in one embodiment of the present disclosure, the generating module 403 is used to:

[0125] Based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, and the example samples of the dialogue response of the human role, the dialogue system of the digital human is learned to obtain an upgraded dialogue system of the digital human;

[0126] Based on the upgraded dialogue system of the digital human, the dialogue reply information for replying to the dialogue question information is generated based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human character.

[0127] Further optionally, in one embodiment of the present disclosure, the intelligent dialogue device 400 of the digital human further includes:

[0128] The parsing module 408 is used to parse the dialogue reply information and obtain the carried tags;

[0129] A synthesis module 409 is used to synthesize the digital human's reply voice based on the emotion tag and the dialogue reply information carried in the dialogue reply information if the tag includes an emotion tag;

[0130] Furthermore, the intelligent dialogue device 400 of the digital human also includes:

[0131] The emotion control module 410 is used to control the facial emotion of the digital human based on the emotion tag carried in the dialogue reply information.

[0132] Further optionally, in one embodiment of the present disclosure, the intelligent dialogue device 400 of the digital human further includes:

[0133] The body control module 411 is used to control the body movements of the digital human in the process of replying the voice based on the body control tag carried in the dialogue reply information if the tag also includes a body control tag.

[0134] The intelligent dialogue device 400 of the digital human of this embodiment can realize the implementation principle and technical effect of the intelligent dialogue of the digital human by adopting the above modules, which is the same as the implementation of the above related method embodiments. For details, please refer to the records of the above related method embodiments, which will not be repeated here.

[0135] In the technical solution disclosed herein, the acquisition, storage and application of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0136] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0137] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0138] like Figure 5As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0139] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0140] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above, such as the above-mentioned methods of the present disclosure. For example, in some embodiments, the above-mentioned methods of the present disclosure may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the above-mentioned methods of the present disclosure described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to perform the above-mentioned methods of the present disclosure in any other appropriate manner (e.g., by means of firmware).

[0141] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0143] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0145] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0146] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0147] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0148] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A digital human intelligent dialogue method, comprising: Obtain the character description information of the digital person; The character description information includes character role information, character information of the character, description information of tags carried in the dialogue response of the character, and example samples of the dialogue response of the character; the tags include emotion tags; Based on the persona description information, obtain a corresponding professional knowledge base; A pre-trained digital human dialogue system is used to generate dialogue response information corresponding to dialogue question information based on the digital human's personality description information and the professional knowledge base; the dialogue response information carries the label; and the digital human dialogue system is implemented based on a large language model.

2. The method according to claim 1, wherein: The tags also include limb control tags.

3. The method according to claim 2, wherein: Based on the persona description information, a corresponding professional knowledge base is obtained, including: Based on the human role information in the human description information, a professional knowledge base of the human role is obtained from a plurality of pre-created professional knowledge bases of known human roles.

4. The method according to claim 3, wherein: Based on the character role information in the character description information, before acquiring the professional knowledge base of the character role from a plurality of pre-created professional knowledge bases of known character roles, the method further includes: For each known human role among a plurality of known human roles, obtaining professional knowledge in a field to which the known human role belongs; Segment the professional knowledge according to preset text length to obtain multiple knowledge segments; Extracting characteristic expressions of each of the knowledge fragments; Based on each of the knowledge fragments and the characteristic expressions of each of the knowledge fragments, a professional knowledge base of the corresponding known human role is constructed.

5. The method according to claim 2, wherein: The pre-trained digital human dialogue system is used to generate dialogue response information corresponding to the dialogue question information based on the digital human's personality description information and the professional knowledge base, including: The digital human dialogue system is adopted to generate the dialogue response information for the dialogue question information based on the human role information, the personality information of the human role, the description information of the label carried in the dialogue response of the human role, the example samples of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, wherein the dialogue response information carries the label.

6. The method according to claim 5, wherein: The digital human dialogue system is used to generate the dialogue response information of the dialogue question information based on the human role information, the personality information of the human role, the description information of the tag carried in the dialogue response of the human role, the example sample of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, including: Based on the human character information, the personality information of the human character, the description information of the tags carried in the dialogue response of the human character, and the example samples of the dialogue response of the human character, the ability of the dialogue system of the digital human to learn the dialogue response information of the human character is controlled; The dialogue system of the digital human after learning is used to generate the dialogue response information for responding to the dialogue question information based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human role.

7. The method according to claim 6, wherein: The dialogue system of the digital human after learning is used to generate the dialogue reply information for replying to the dialogue question information based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human role, including: Using the learned dialogue system of the digital human, based on the dialogue question information and the historical dialogue information, to obtain relevant knowledge fragments from the professional knowledge base of the human character; The dialogue system of the digital human after learning is used to generate the dialogue reply information for replying to the dialogue question information based on the dialogue question information, the historical dialogue information, and the relevant knowledge fragments.

8. The method according to claim 5, wherein: The digital human dialogue system is used to generate the dialogue response information of the dialogue question information based on the human role information, the personality information of the human role, the description information of the tag carried in the dialogue response of the human role, the example sample of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, including: Based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, and the example samples of the dialogue response of the human role, the dialogue system of the digital human is learned to obtain an upgraded dialogue system of the digital human; Based on the upgraded dialogue system of the digital human, the dialogue reply information for replying to the dialogue question information is generated based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human character.

9. The method according to any one of claims 5 to 8, wherein: After the dialogue system of the digital human generates the dialogue response information of the dialogue question information based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, the example samples of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, the method further includes: Parsing the conversation reply information to obtain the carried tags; If the tag includes an emotional tag, synthesizing the digital human's reply voice based on the emotional tag and the dialogue reply information carried in the dialogue reply information; Furthermore, the method further comprises: Based on the emotion tag carried in the dialogue reply information, the facial expression of the digital human is controlled.

10. The method according to claim 9, wherein: After the dialogue system of the digital human generates the dialogue response information of the dialogue question information based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, the example samples of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, the method further includes: If the tag also includes a body control tag, based on the body control tag carried in the dialogue reply information, the body movement of the digital human in the process of replying the voice is controlled.

11. An intelligent conversation device for a digital human, comprising: A character acquisition module is used to obtain character description information of a digital human; The character description information includes character role information, character information of the character, description information of tags carried in the dialogue response of the character, and example samples of the dialogue response of the character; the tags include emotion tags; A knowledge base acquisition module, used to acquire a corresponding professional knowledge base based on the persona description information; A generation module is used to use a pre-trained digital human dialogue system to generate dialogue response information corresponding to dialogue question information based on the digital human's personality description information and the professional knowledge base; the dialogue response information carries the label; the digital human dialogue system is implemented based on a large language model.

12. The device according to claim 11, wherein The tags also include limb control tags.

13. The device according to claim 12, wherein: The knowledge base acquisition module is used to: Based on the human role information in the human description information, a professional knowledge base of the human role is obtained from a plurality of pre-created professional knowledge bases of known human roles.

14. The device according to claim 13, wherein: The device also includes: A knowledge acquisition module, for acquiring, for each of a plurality of known human roles, professional knowledge in a field to which the known human role belongs; A segmentation module is used to segment the professional knowledge according to a preset text length to obtain multiple knowledge segments; An extraction module, used to extract the characteristic expression of each of the knowledge fragments; A construction module is used to construct a professional knowledge base of the corresponding known human role based on each of the knowledge fragments and the characteristic expressions of each of the knowledge fragments.

15. The device according to claim 12, wherein: The generating module is used for: The digital human dialogue system is adopted to generate the dialogue response information of the dialogue question information based on the human role information, the personality information of the human role, the description information of the label carried in the dialogue response of the human role, the example samples of the dialogue response of the human role and the professional knowledge base of the human role, and with reference to the historical dialogue information, wherein the dialogue response information carries the label.

16. The device according to claim 15, wherein: The generating module is used for: Based on the human character information, the personality information of the human character, the description information of the tags carried in the dialogue response of the human character, and the example samples of the dialogue response of the human character, the ability of the dialogue system of the digital human to learn the dialogue response information of the human character is controlled; The dialogue system of the digital human after learning is used to generate the dialogue response information for responding to the dialogue question information based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human role.

17. The device according to claim 16, wherein: The generating module is used for: Using the learned dialogue system of the digital human, based on the dialogue question information and the historical dialogue information, to obtain relevant knowledge fragments from the professional knowledge base of the human character; The dialogue system of the digital human after learning is used to generate the dialogue reply information for replying to the dialogue question information based on the dialogue question information, the historical dialogue information, and the relevant knowledge fragments.

18. The device according to claim 15, wherein: The generating module is used for: Based on the human role information, the personality information of the human role, the description information of the tags carried in the dialogue response of the human role, and the example samples of the dialogue response of the human role, the dialogue system of the digital human is learned to obtain an upgraded dialogue system of the digital human; Based on the upgraded dialogue system of the digital human, the dialogue reply information for replying to the dialogue question information is generated based on the dialogue question information, the historical dialogue information and the professional knowledge base of the human character.

19. The device according to any one of claims 15 to 18, wherein: The device also includes: A parsing module, used to parse the conversation reply information and obtain the carried tags; A synthesis module, for synthesizing the digital person's reply voice based on the emotion tag and the dialogue reply information carried in the dialogue reply information if the tag includes an emotion tag; Furthermore, the device also includes: The emotion control module is used to control the facial expression of the digital human based on the emotion tag carried in the dialogue reply information.

20. The device according to claim 19, wherein The device also includes: The body control module is used to control the body movements of the digital human in the process of replying the voice based on the body control tag carried in the dialogue reply information if the tag also includes the body control tag.

21. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 10.

22. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-10.

23. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 10.

Images