A digital human expression driving method, system and program product

Abstract

The present invention belongs to the field of human-computer interaction technology, and specifically discloses a method, system and program product for driving digital human expressions. By adding emotion tags and emotion strength when obtaining answer texts, and effectively weighted fusion of phoneme visemes and emotion visemes during viseme fusion, the expression effect control of digital humans that is compatible with emotions and pronunciation is realized. The emotion tags drive the facial expressions of digital humans on the one hand, so that they can show appropriate emotions while expressing content, thereby improving the vividness of digital human interaction and the accuracy of emotional expression. The present invention solves the problem in the prior art that digital human expressions are only based on phoneme changes and lack emotional expression, significantly enhances the emotional expression ability of digital humans, makes them more authentic and friendly, and improves the user's interactive experience. It is suitable for digital human interactive application scenarios such as virtual assistants and virtual anchors that have high requirements for expressions and emotions.

Description

Technical Field

[0001] The present invention belongs to the technical field of human-computer interaction, and in particular relates to a digital human expression driving method, system and program product. Background Art

[0002] With the rapid development of artificial intelligence technology, the application of large language models combined with digital human technology to complete role-playing is becoming more and more widespread, especially in the fields of virtual assistants, games, entertainment, etc. 3D digital humans have become one of the main forms of interaction with users. Natural interaction elements such as 3D digital humans' expressions, movements and tones are crucial to improving user experience. Visual pixels, as a technical solution to drive the facial morphology of 3D digital humans, provide a mechanism to display the facial morphology of virtual characters 3D digital humans through image or graphic changes.

[0003] At present, viseme technology usually generates corresponding visemes based on the acquired phonemes, and adjusts the visemes in combination with the voice pitch to drive the facial morphology of 3D digital humans. For example, when a large language model is generated, the system can adjust the facial expressions of digital humans according to the pitch of the voice, making the expression of the virtual character more vivid and natural.

[0004] However, existing viseme technology still has certain limitations. It mainly relies on changes in input phonemes, such as changes in the pitch of voice, to drive the expressions of 3D digital humans. This means that the response of the viseme system is limited to adjustment through physical signals or input conditions, and cannot further adjust the expression according to the emotions, attitudes or emotions contained in the reply text generated by the large language model (such as GPT, etc.). In other words, although the large language model can accurately generate complex and emotional dialogue content, the digital human cannot appropriately adjust its expression according to the potential emotional changes in the model's reply content, which causes the virtual character's expression to often appear dull or lack emotional depth, and cannot vividly reflect the details of emotional fluctuations. Summary of the invention

[0005] The purpose of the present invention is to provide a digital human expression driving method, system and program product to solve the above problems existing in the prior art.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, a method for driving digital human expression is provided, comprising:

[0008] Obtain the answer text and the emotion label and emotion intensity corresponding to the answer text;

[0009] Performing text-to-speech processing based on the answer text to generate corresponding speech content and determine a phoneme sequence of the speech content, wherein the phoneme sequence includes a phoneme vector corresponding to each time point;

[0010] Using a preset phoneme-viseme conversion table to perform mapping conversion processing on the phoneme sequence to obtain a corresponding phoneme-viseme sequence, wherein the phoneme-viseme sequence includes a first expression vector corresponding to each time point;

[0011] Extracting a corresponding emotion visual element from a preset emotion visual element set according to the emotion tag, wherein the emotion visual element includes a corresponding second expression vector;

[0012] Perform viseme fusion processing based on the phoneme viseme sequence, the emotion viseme and the emotion intensity to obtain a fused viseme sequence, wherein the fused viseme sequence includes a fused expression vector corresponding to each time point;

[0013] During the display of the digital human, the fused expression vectors of the corresponding time points of the fused visual element sequence are used to control the expression of the digital human.

[0014] In a possible design, before obtaining the answer text and the emotion label and emotion intensity corresponding to the answer text, the method further includes:

[0015] Get input text;

[0016] The input text is input into a preset large language model for natural language processing, and the large language model outputs the answer text as well as the emotion label and emotion intensity corresponding to the answer text.

[0017] In a possible design, outputting the answer text and the emotion label and emotion intensity corresponding to the answer text through the large language model includes:

[0018] A pre-built prompt word Prompt is provided to the large language model, so that the large language model outputs the emotion label and emotion intensity corresponding to the answer text while outputting the answer text in response to the prompt word Prompt.

[0019] In a possible design, obtaining input text includes:

[0020] Get the dialog text entered by the user and use the dialog text entered by the user as the input text;

[0021] Alternatively, the conversation voice input by the user is obtained, and speech recognition processing is performed on the conversation voice input by the user to obtain a corresponding conversation text, and the conversation text is used as the input text.

[0022] In one possible design, the phoneme-viseme conversion table includes a plurality of phoneme vectors and a first expression vector after each phoneme vector is mapped and converted, and the emotion viseme set includes a plurality of second expression vectors, and each second expression vector is associated with a corresponding emotion label.

[0023] In a possible design, the viseme fusion process is performed based on the phoneme viseme sequence, the emotion viseme and the emotion intensity to obtain the fused viseme sequence, including:

[0024] Substitute the first expression vector, the second expression vector and the emotion intensity corresponding to each time point into the preset visual element fusion formula for fusion calculation to obtain the fused expression vector corresponding to each time point. The visual element fusion formula is:

[0025] V f (t) = I ph *(1-I em )*V ph (t)+I em *(1-I ph )*V em

[0026] Among them, t represents the corresponding time point, V f (t) represents the fused expression vector corresponding to time point t, V ph (t) represents the first expression vector corresponding to time point t, V em represents the second expression vector, I ph Indicates the set phoneme intensity, I em Indicates emotional intensity;

[0027] The fused expression vectors corresponding to each time point are used to form a fused visual element sequence.

[0028] In a possible design, when the fused expression vectors at corresponding time points of the fused visual element sequence are used to control the expression of the digital human, the method further includes: synchronously outputting the voice content.

[0029] In a second aspect, a digital human expression driving system is provided, comprising a text acquisition unit, a language processing unit, a phoneme generation unit, a viseme mapping unit, a viseme extraction unit, a viseme fusion unit and an expression control unit, wherein:

[0030] A text acquisition unit, used for acquiring input text;

[0031] A language processing unit, used to input the input text into a preset large language model for natural language processing, and output a reply text and a sentiment label and sentiment intensity corresponding to the reply text through the large language model;

[0032] A phoneme generation unit, configured to perform text-to-speech processing based on the answer text, generate corresponding speech content, and determine a phoneme sequence of the speech content, wherein the phoneme sequence includes a phoneme vector corresponding to each time point;

[0033] a viseme mapping unit, configured to perform mapping conversion processing on the phoneme sequence using a preset phoneme-viseme conversion table to obtain a corresponding phoneme-viseme sequence, wherein the phoneme-viseme sequence includes a first expression vector corresponding to each time point;

[0034] A viseme extraction unit, configured to extract a corresponding emotion viseme from a preset emotion viseme set according to the emotion tag, wherein the emotion viseme includes a corresponding second expression vector;

[0035] A viseme fusion unit, configured to perform viseme fusion processing based on a phoneme viseme sequence, an emotion viseme, and an emotion intensity to obtain a fused viseme sequence, wherein the fused viseme sequence includes a fused expression vector corresponding to each time point;

[0036] The expression control unit is used to control the expression of the digital human by using the fused expression vector of the corresponding time point of the fused visual element sequence during the display process of the digital human.

[0037] In a third aspect, a digital human expression driving system is provided, comprising:

[0038] A memory for storing instructions;

[0039] A processor is used to read the instructions stored in the memory and execute any one of the methods described in the first aspect according to the instructions.

[0040] In a fourth aspect, a computer-readable storage medium is provided, wherein instructions are stored on the computer-readable storage medium, and when the instructions are executed on a computer, the computer executes any one of the methods described in the first aspect. In addition, a computer program product is provided, and when the computer program product is executed on a computer, the computer executes any one of the methods described in the first aspect.

[0041] Beneficial effects: The present invention achieves the control of digital human expression effects that are compatible with emotions and pronunciation by adding emotion tags and emotion intensity when obtaining the answer text, and effectively weighted fusion of phoneme visemes and emotion visemes during the viseme fusion process. On the one hand, the emotion tags drive the facial expressions of digital humans, so that they can show appropriate emotions while expressing the content, thereby improving the vividness of digital human interaction and the accuracy of emotional expression. The present invention solves the problem in the prior art that digital human expressions are only based on phoneme changes and lack emotional expression, significantly enhances the emotional expression ability of digital humans, makes them more authentic and friendly, and improves the user's interactive experience. It is suitable for digital human interaction application scenarios such as virtual assistants and virtual anchors that have high requirements for expressions and emotions. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0043] Figure 1 This is a schematic diagram of the steps of the method in Example 1 of the present invention;

[0044] Figure 2 This is a schematic diagram of the structure of the system in Example 2 of the present invention;

[0045] Figure 3 Schematic diagram of the system structure in Example 3 of the present invention. DETAILED DESCRIPTION

[0046] It should be noted that the description of these embodiments is used to help understand the present invention, but does not constitute a limitation of the present invention. The specific structures and functional details disclosed herein are only used to describe the exemplary embodiments of the present invention. However, the present invention can be embodied in many alternative forms, and it should not be understood that the present invention is limited to the embodiments set forth herein.

[0047] It should be understood that unless otherwise clearly specified and limited, the corresponding terms should be understood in a broad sense. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; it can be a direct connection, or an indirect connection through an intermediate medium, or it can be the internal communication of two elements. For ordinary technicians in this field, the specific meanings of the above terms in the embodiments can be understood according to specific circumstances.

[0048] In the following description, certain details are provided to facilitate a complete understanding of the example embodiments. However, it will be appreciated by those of ordinary skill in the art that the example embodiments may be implemented without these certain details. For example, devices may be shown in block diagrams to avoid obscuring the examples with unnecessary details. In other embodiments, well-known processes, structures, and techniques may not be shown in unnecessary detail to avoid obscuring the embodiments.

[0049] Embodiment 1:

[0050] This embodiment provides a digital human expression driving method, which can be applied to corresponding intelligent interaction systems, such as Figure 1 As shown, the method comprises the following steps:

[0051] S1. Get input text.

[0052] In specific implementation, the system may obtain the dialogue text input by the user and use the dialogue text input by the user as the input text. Alternatively, the system may obtain the dialogue voice input by the user and perform speech recognition processing on the dialogue voice input by the user, such as inputting the dialogue voice into an ASR (Automatic Speech Recognition) module, converting the voice into corresponding dialogue text through the ASR module, and using the dialogue text as the input text.

[0053] S2. Input the input text into a preset large language model for natural language processing, and output the answer text and the emotion label and emotion intensity corresponding to the answer text through the large language model.

[0054] In specific implementation, the system inputs the input text into a preset large language model (such as GPT, etc.) for natural language processing, and outputs the answer text and the emotional label and emotional intensity corresponding to the answer text through the large language model. When using the large language model for natural language processing, the large language model can be provided with a pre-constructed prompt word Prompt, so that the large language model responds to the prompt word Prompt while outputting the answer text. At the same time, it outputs the emotional label and emotional intensity corresponding to the answer text, that is, each answer text automatically adds an emotional label and emotional intensity that expresses the emotion (such as "<happy, 0.6>", <happy> is the emotional label, and the emotional intensity is 0.6), so as to help the subsequent steps to accurately generate 3D digital human expressions that conform to the context and emotions. The emotional labels can be set as follows: <normal>, <happy>, <sad>, <encouraged>, <surprised>, <gentle>, <angry>, <helpless>, <gratified>, <suspicious>. Exemplarily:

[0055] For example, the prompt word Prompt is set as:

[0056] "You are an emotional virtual assistant, and users will ask you various questions or share stories. Please generate an answer based on the user input, and attach the emotional label and emotional intensity of each sentence after the answer. The emotional label needs to accurately reflect the emotion in the answer, and the emotional intensity needs to accurately reflect the strength of the emotion in the answer, with a value range of 0 to 1. Emotional labels are selected from the following emotions: <normal>, <happy>, <sad>, <encouraged>, <surprised>, <gentle>, <angry>, <helpless>, <relieved>, <suspicious>. Please add the emotional label and emotional intensity at the end of each sentence of the answer. The format is: [answer sentence] <emotional label, emotional intensity>".

[0057] Example input text:

[0058] “I finally completed a big project today and I feel very accomplished!…”.

[0059] For example, the output of the large language model is:

[0060] "I am also very happy to hear about your achievement, <happy, 0.6>. You have really put in a lot of effort and deserve such a result, <encouraged, 0.7>. Keep up this positive attitude, I believe you will get better and better, <gratified, 0.2>..."

[0061] That is, the output of the large language model can be divided into two parts: answer information and emotional information. These two parts of information will be used to generate appropriate expressions. For example, its output can generate corresponding json format information:

[0062]

[0063]

[0064] The emotional label part will be used for subsequent emotional visual element processing to adjust the digital human's expression and ensure that each sentence carries the appropriate emotion. The answer text part will be used for subsequent speech synthesis by the user to generate speech that fits the context.

[0065] S3. Perform text-to-speech processing based on the answer text to generate corresponding speech content and determine the phoneme sequence of the speech content, wherein the phoneme sequence includes the phoneme vectors corresponding to each time point.

[0066] In specific implementation, the answer text can be processed into speech through text-to-speech technology (generating speech that matches the tone and emotion through text-to-speech technology), such as inputting the answer text into a TTS (Text-to-Speech) module to generate the corresponding speech content and determine the phoneme sequence of the speech content, which includes the phoneme vectors corresponding to each time point.

[0067] S4. Use a preset phoneme-viseme conversion table to perform mapping conversion processing on the phoneme sequence to obtain a corresponding phoneme-viseme sequence, wherein the phoneme-viseme sequence includes a first expression vector corresponding to each time point.

[0068] In specific implementation, the system can use a preset phoneme-viseme conversion table (including a number of phoneme vectors and the first expression vector after each phoneme vector is mapped and converted) to map and convert the phoneme sequence to obtain a corresponding phoneme-viseme sequence, which includes the first expression vector corresponding to each time point. Phoneme-viseme is facial expression information generated based on the change of speech phonemes, and is mainly used to drive the basic mouth shape and pronunciation expression of the digital human face. For example, the phoneme-viseme of "O" will make the lips round, while the phoneme-viseme of "E" will stretch the corners of the mouth.

[0069] S5. Extract corresponding emotion visual elements from a preset emotion visual element set according to the emotion tag, wherein the emotion visual element includes a corresponding second expression vector.

[0070] In specific implementation, the system can extract corresponding emotion visual elements from the preset emotion visual element set according to the emotion tag, and the emotion visual element includes the corresponding second expression vector. The emotion visual element set includes a number of second expression vectors, and each second expression vector is associated with a corresponding emotion tag. Emotion visual elements are expression information generated according to the emotion tags in the text, which are used to express emotions. For example, <happy, 0.6> represents the emotion of "happy" with an intensity of 60%. Emotion visual elements usually affect the expressions of areas such as eyes and eyebrows, making the digital human show more vivid emotional reactions.

[0071] S6. Perform viseme fusion processing based on the phoneme-viseme sequence, the emotion viseme and the emotion intensity to obtain a fused viseme sequence, wherein the fused viseme sequence includes a fused expression vector corresponding to each time point.

[0072] In specific implementation, the goal of viseme fusion processing is to integrate the influence of phoneme visemes and emotion visemes, so that the face of the digital human can reflect the pronunciation changes and convey the emotional level. For example, when expressing the emotion of "happy", the system must not only generate the corresponding smiling expression, but also maintain the phoneme expression of the mouth to ensure the natural match between voice and expression. When using phoneme viseme sequence, emotion viseme and emotion intensity for viseme fusion processing: substitute the first expression vector, the second expression vector and the emotion intensity corresponding to each time point into the preset viseme fusion formula for fusion calculation, and obtain the fused expression vector corresponding to each time point. The viseme fusion formula is:

[0073] V f (t) = I ph *(1-I em )*V ph (t)+I em *(1-I ph )*V em

[0074] Among them, t represents the corresponding time point, V f (t) represents the fused expression vector corresponding to time point t, V ph (t) represents the first expression vector corresponding to time point t, V em represents the second expression vector, I ph Indicates the set phoneme intensity, I em Indicates the intensity of emotion. Then, based on the fused expression vectors corresponding to each time point, a fused viseme sequence is formed. The result of viseme fusion is used to control the facial expressions of digital humans, and to generate natural facial expressions that meet the requirements of emotion and pronunciation in real time, so that the expressions of virtual characters are coordinated with the content of user interaction.

[0075] S7. During the display of the digital human, the fused expression vector of the corresponding time point of the fused visual element sequence is used to control the expression of the digital human.

[0076] In specific implementation, the system outputs voice content and displays the digital human simultaneously, and during the display of the digital human, the fused expression vector of the corresponding time point of the fused visual element sequence is used to control the expression of the digital human. Finally, the digital human gives feedback to the user through answers with expressions and tones, completing an interactive process with emotional and natural expressions.

[0077] The method of this embodiment solves the problem in the prior art that the expressions of digital humans are only based on phoneme changes and lack emotional expression, significantly enhances the emotional expression ability of digital humans, makes them more realistic and friendly, and improves the user's interactive experience. It is suitable for digital human interactive application scenarios such as virtual assistants and virtual anchors that have high requirements for expressions and emotions.

[0078] Embodiment 2:

[0079] This embodiment provides a digital human expression driving system. Figure 2 As shown, it includes a text acquisition unit, a language processing unit, a phoneme generation unit, a viseme mapping unit, a viseme extraction unit, a viseme fusion unit and an expression control unit, wherein:

[0080] A text acquisition unit, used for acquiring input text;

[0081] A language processing unit, used to input the input text into a preset large language model for natural language processing, and output a reply text and a sentiment label and sentiment intensity corresponding to the reply text through the large language model;

[0082] A phoneme generation unit, configured to perform text-to-speech processing based on the answer text, generate corresponding speech content, and determine a phoneme sequence of the speech content, wherein the phoneme sequence includes a phoneme vector corresponding to each time point;

[0083] a viseme mapping unit, configured to perform mapping conversion processing on the phoneme sequence using a preset phoneme-viseme conversion table to obtain a corresponding phoneme-viseme sequence, wherein the phoneme-viseme sequence includes a first expression vector corresponding to each time point;

[0084] A viseme extraction unit, configured to extract a corresponding emotion viseme from a preset emotion viseme set according to the emotion tag, wherein the emotion viseme includes a corresponding second expression vector;

[0085] A viseme fusion unit, configured to perform viseme fusion processing based on a phoneme viseme sequence, an emotion viseme, and an emotion intensity to obtain a fused viseme sequence, wherein the fused viseme sequence includes a fused expression vector corresponding to each time point;

[0086] The expression control unit is used to control the expression of the digital human by using the fused expression vector of the corresponding time point of the fused visual element sequence during the display process of the digital human.

[0087] Embodiment 3:

[0088] This embodiment provides a digital human expression driving system. Figure 3 As shown, at the hardware level, it includes:

[0089] Data interface, used to establish data connection between the processor and the user end;

[0090] A memory for storing instructions;

[0091] The processor is used to read the instructions stored in the memory and execute the digital human expression driving method in embodiment 1 according to the instructions.

[0092] Optionally, the system further includes an internal bus, through which the processor, the memory and the data interface can be interconnected, and the internal bus can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc.

[0093] The memory may include, but is not limited to, random access memory (RAM), read only memory (ROM), flash memory, first input first output (FIFO) and / or first in last out (FILO), etc. The processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0094] Embodiment 4:

[0095] This embodiment provides a computer-readable storage medium, on which instructions are stored. When the instructions are executed on a computer, the computer executes the digital human expression driving method in Embodiment 1. The computer-readable storage medium refers to a carrier for storing data, which may include but is not limited to a floppy disk, an optical disk, a hard disk, a flash memory, a USB flash drive and / or a memory stick, etc. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0096] This embodiment also provides a computer program product, which, when running on a computer, executes the digital human expression driving method in Embodiment 1. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable devices.

[0097] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A digital human expression driving method, characterized in that: include: Obtain the answer text and the emotion label and emotion intensity corresponding to the answer text; Performing text-to-speech processing based on the answer text to generate corresponding speech content and determine a phoneme sequence of the speech content, wherein the phoneme sequence includes a phoneme vector corresponding to each time point; Using a preset phoneme-viseme conversion table to perform mapping conversion processing on the phoneme sequence to obtain a corresponding phoneme-viseme sequence, wherein the phoneme-viseme sequence includes a first expression vector corresponding to each time point; Extracting a corresponding emotion visual element from a preset emotion visual element set according to the emotion tag, wherein the emotion visual element includes a corresponding second expression vector; Perform viseme fusion processing based on the phoneme viseme sequence, the emotion viseme and the emotion intensity to obtain a fused viseme sequence, wherein the fused viseme sequence includes a fused expression vector corresponding to each time point; During the display of the digital human, the fused expression vectors of the corresponding time points of the fused visual element sequence are used to control the expression of the digital human.

2. A digital human expression driving method according to claim 1, characterized in that: Before obtaining the answer text and the emotion label and emotion intensity corresponding to the answer text, the method further includes: Get input text; The input text is input into a preset large language model for natural language processing, and the large language model outputs the answer text as well as the emotion label and emotion intensity corresponding to the answer text.

3. A digital human expression driving method according to claim 2, characterized in that: Outputting the answer text and the emotion label and emotion intensity corresponding to the answer text through the large language model includes: A pre-built prompt word Prompt is provided to the large language model, so that the large language model outputs the emotion label and emotion intensity corresponding to the answer text while outputting the answer text in response to the prompt word Prompt.

4. A digital human expression driving method according to claim 2, characterized in that: The step of obtaining the input text comprises: Get the dialog text entered by the user and use the dialog text entered by the user as the input text; Alternatively, the conversation voice input by the user is obtained, and speech recognition processing is performed on the conversation voice input by the user to obtain a corresponding conversation text, and the conversation text is used as the input text.

5. A digital human expression driving method according to claim 1, characterized in that: The phoneme-viseme conversion table includes a plurality of phoneme vectors and a first expression vector after each phoneme vector is mapped and converted. The emotion viseme set includes a plurality of second expression vectors, and each second expression vector is associated with a corresponding emotion label.

6. A digital human expression driving method according to claim 1, characterized in that: The viseme fusion process is performed based on the phoneme viseme sequence, the emotion viseme and the emotion intensity to obtain a fused viseme sequence, including: Substitute the first expression vector, the second expression vector and the emotion intensity corresponding to each time point into the preset visual element fusion formula for fusion calculation to obtain the fused expression vector corresponding to each time point. The visual element fusion formula is: V f (t)=I ph *(1-I em )*V ph (t)+I em *(1-I ph )*V em Among them, t represents the corresponding time point, V f (t) represents the fused expression vector corresponding to time point t, V ph (t) represents the first expression vector corresponding to time point t, V em represents the second expression vector, I ph Indicates the set phoneme intensity, I em Indicates emotional intensity; The fused expression vectors corresponding to each time point are used to form a fused visual element sequence.

7. A digital human expression driving method according to claim 1, characterized in that: When the fused expression vectors at the corresponding time points of the fused visual element sequence are used to control the expression of the digital human, the method further includes: synchronously outputting the voice content.

8. A digital human expression driving system, characterized in that: It includes a text acquisition unit, a language processing unit, a phoneme generation unit, a viseme mapping unit, a viseme extraction unit, a viseme fusion unit and an expression control unit, wherein: A text acquisition unit, used for acquiring input text; A language processing unit, used to input the input text into a preset large language model for natural language processing, and output a reply text and a sentiment label and sentiment intensity corresponding to the reply text through the large language model; A phoneme generation unit, configured to perform text-to-speech processing based on the answer text, generate corresponding speech content, and determine a phoneme sequence of the speech content, wherein the phoneme sequence includes a phoneme vector corresponding to each time point; a viseme mapping unit, configured to perform mapping conversion processing on the phoneme sequence using a preset phoneme-viseme conversion table to obtain a corresponding phoneme-viseme sequence, wherein the phoneme-viseme sequence includes a first expression vector corresponding to each time point; A viseme extraction unit, configured to extract a corresponding emotion viseme from a preset emotion viseme set according to the emotion tag, wherein the emotion viseme includes a corresponding second expression vector; A viseme fusion unit, configured to perform viseme fusion processing based on a phoneme viseme sequence, an emotion viseme, and an emotion intensity to obtain a fused viseme sequence, wherein the fused viseme sequence includes a fused expression vector corresponding to each time point; The expression control unit is used to control the expression of the digital human by using the fused expression vector of the corresponding time point of the fused visual element sequence during the display process of the digital human.

9. A digital human expression driving system, characterized in that: include: A memory for storing instructions; A processor is used to read the instructions stored in the memory and execute the digital human expression driving method described in any one of claims 1 to 7 according to the instructions.

10. A computer program product, characterized in that When the computer program product is run on a computer, the digital human expression driving method described in any one of claims 1 to 7 is executed.

Images