Method and device for improving fluency of facial expressions and actions of digital human and storage medium

Through the GR-TMCNN model combined with convolutional and recurrent neural network, dynamic adjustment of digital human facial expressions is achieved, which solves the problem of stiff expressions in traditional digital human technology, improves fluency and realism, and reduces the computational complexity.

CN119091012APending Publication Date: 2024-12-06JIANGSU ZHIHENG INFORMATION TECH SERVICES CO LTD
View PDF 0 Cites 3 Cited by

Patent Information

Application Number
CN202411107520.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-13
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Traditional digital human technology cannot achieve dynamic adjustment and natural movement of facial expressions, resulting in stiff expressions and difficult to meet the needs of highly realistic human interactions.

Method used

The GR-TMCNN model is adopted, combined with the convolutional network and the recurrent neural network, and the eigenvalue of the Mel frequency cepspectral coefficient and text emotions of the audio data can be used to achieve the precise linkage of digital human lip shape and facial movements, using the triangular convolution kernel and Post-mask convolution improves the generalization and computational efficiency of the model.

Benefits of technology

It improves the fluency and realism of digital human facial expressions and movements, enhances emotional expression ability, reduces calculation complexity, and ensures efficient and stable processing performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091012A_ABST
    Figure CN119091012A_ABST
Patent Text Reader

Abstract

The invention provides a method for improving fluency of facial expressions and actions of a digital person. The method comprises the following steps: acquiring an audio sent to the digital person and a text corresponding to the audio; extracting an MFCC feature value of the audio, and performing text emotion recognition on the text to obtain a text emotion; inputting the MFCC feature value and the text emotion into a pre-trained GR-TMCNN model to obtain a lip-shaped action and a face action of the digital human; sending the lip and face actions of the digital person to a UE engine to drive the digital person; wherein the GR-TMCNN model comprises a convolutional network and a recurrent neural network. According to the GR-TMCNN model, the convolutional network and the recurrent neural network are combined, the convolutional network realizes feature extraction of multi-dimensional data, the recurrent neural network realizes calculation related to text and audio time sequence, and the model can perform feature extraction on the multi-dimensional data while retaining time sequence features through combination of the convolutional network and the recurrent neural network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method, a device and a storage medium for improving the fluency of digital human facial expression movements, and belongs to the field of deep learning of speech signal processing and the field of multimodal digital human driving. Background Art

[0002] With the rapid development of semiconductor technology and the continuous expansion of the scale of deep learning model parameters, the field of artificial intelligence has ushered in unprecedented changes. In the field of speech processing, advanced speech engines such as VIT2 can not only generate high-quality audio data, but also give the audio rich emotional colors based on training data, greatly improving the naturalness of human-computer interaction. At the same time, large language models such as GLM4 have demonstrated powerful text understanding and generation capabilities, and can provide reasonable answers to complex questions based on existing knowledge.

[0003] However, despite many advances in the field of digital human technology, traditional methods still have significant shortcomings. For example, the digital human generation system based on wav2lib technology mainly relies on video processing of static images and cannot achieve dynamic movement adjustment based on real-time instructions. Although the digital human based on Unreal Engine 5 (UE5) can simulate lip movement, due to the lack of fine control of facial muscle movement, its expression appears stiff and unnatural, which makes it difficult to meet users' needs for highly realistic and humanized interactive experience.

[0004] To solve this problem, we try to explore some new technologies and methods in the field of deep learning. Summary of the invention

[0005] The purpose of the present invention is to overcome the deficiencies in the prior art, provide a method for improving the fluency of digital human facial expression movements, improve the accuracy and real-time performance of digital human lip shape movement recognition, and achieve precise linkage between digital human facial muscles and lip shape.

[0006] To achieve the above object, the present invention is implemented by adopting the following technical solutions:

[0007] In a first aspect, the present invention provides a method for improving the fluency of digital human facial expressions and movements, comprising:

[0008] Step 1: Obtain the audio data sent to the digital human and the text corresponding to the audio data;

[0009] Step 2: extracting the Mel-frequency cepstral coefficient MFCC feature value of the audio data, performing text emotion recognition on the text, and obtaining the text emotion;

[0010] Step 3: Input the MFCC feature values ​​and the text emotions into the pre-trained GR-TMCNN model to obtain the lip movements and facial movements of the digital human;

[0011] Step 4: Send the digital human's lip movements and facial movements to the digital human, thereby driving the digital human;

[0012] Wherein, the GR-TMCNN model includes: a convolutional network and a recurrent neural network;

[0013] The step 3 comprises:

[0014] Divide the MFCC feature value into multiple tensor slices according to the time frame;

[0015] Input tensor slices into the convolutional network to extract MFCC temporal feature vectors;

[0016] Input the MFCC temporal feature vector into a recurrent neural network to obtain a corresponding lip shape result;

[0017] According to the order of tensor slices, the lip shape results corresponding to each slice are combined into the lip shape movement of the digital human;

[0018] The two-dimensional data consisting of the text emotion and lip shape movement is input into a recurrent neural network to obtain the facial movement of the digital human.

[0019] Furthermore, the two-dimensional data consisting of the text emotion and the lip shape movement is input into a recurrent neural network to obtain the facial movement of the digital human, including:

[0020] Inputting the two-dimensional data consisting of the text emotion and lip shape movement into a recurrent neural network in the form of a preset time length data stream;

[0021] For any preset duration data stream, the recurrent neural network outputs the confidence of the facial muscle movements corresponding to each facial expression; selects the n facial expressions with the highest confidence, and weights the facial muscle movements corresponding to each facial expression according to the weights of the n facial expressions to obtain the facial movement corresponding to the data, where n is a preset value.

[0022] Furthermore, the facial expression includes any expression in a preset expression set, and each facial expression has a corresponding facial muscle movement.

[0023] Furthermore, the convolution network includes: triangular convolution, and the convolution kernel of the triangular convolution has a length in the vertical direction greater than its length in the horizontal direction.

[0024] Furthermore, the convolution rule of the triangular convolution is: in the convolution operation, the triangular convolution kernel slides on the input data in a preset manner, and the convolution point spatially aligned with the center of the triangular convolution kernel is the current convolution point; in the convolution calculation, the product of the data of each point in the triangular convolution kernel and its corresponding convolution point is first calculated, and then all the products are added together to obtain the convolution result of the current convolution point.

[0025] Furthermore, the convolution network also includes: a post-mask convolution; the post-mask convolution is used to randomly decide whether to perform mask processing on the last element in the convolution kernel in this convolution calculation before each convolution operation; in response to the decision to perform mask processing, the last element in the convolution kernel is recorded as 0.

[0026] Furthermore, the lip shape result includes any lip shape in a lip shape set, and the lip shape set includes: sil, PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou.

[0027] Furthermore, the RoBERTa model is used to perform text emotion recognition on the text, and the text emotions include: anger, disgust, fear, happiness, sadness and surprise.

[0028] In a second aspect, the present invention provides an intelligent device, comprising: a processor and a memory connected to the processor; a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method for improving the fluency of digital human facial expression movements as described in the first aspect are implemented.

[0029] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method for improving the fluency of digital human facial expression movements as described in the first aspect.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] (1) The GR-TMCNN model used in the method for improving the fluency of digital human facial expressions and movements provided by the present invention combines a convolutional network and a recurrent neural network. The convolutional network realizes feature extraction of multidimensional data, and the recurrent neural network realizes calculation related to the temporal sequence of text and audio. The combination of the two enables the model to extract features from multidimensional data while retaining temporal features;

[0032] (2) The method for improving the fluency of digital human facial expressions and movements provided by the present invention innovatively develops the GR-TMCNN (Gated Recurrent Triangle Mask Convolutional Neural Network) algorithm. Based on the gated recurrent unit GRU, the algorithm introduces post-mask convolution and triangular convolution kernel. The post-mask convolution randomly decides whether to ignore the contribution of the last element in the convolution kernel in the convolution calculation, thereby improving the generalization of the model. The triangular convolution kernel is longer in the vertical direction, which can better capture the temporal characteristics of audio data, and is reduced in the horizontal direction to control the amount of calculation. At the same time, such a triangular convolution kernel enhances the convolution of the previous frame data and reduces the convolution of the feature value data of the next frame, so that the model can learn more knowledge in the previous frame data and minimize the influence of the next frame data on the current frame while maintaining association with the next frame data.

[0033] (3) The GR-TMCNN model used in the method for improving the fluency of digital human facial expressions and movements provided by the present invention has also made innovations in the convolution method. When performing convolution operations on edge points in an image, the convolution method only calculates some of the points that are close to each other, ensuring that the data points that are related to the temporal sequence are calculated without missing the eigenvalue attributes, extracting the MFCC features related to the temporal sequence, and keeping the shape consistent before convolution;

[0034] (4) The method provided by the present invention for improving the fluency of digital human facial expression movements utilizes deep learning technology, combined with lip shape and emotion recognition algorithms, to achieve accurate linkage between digital human facial muscles and lip shape; by training a data set containing a variety of micro-expressions, the model can recognize and drive the corresponding facial muscles to perform natural and smooth movements, thereby enhancing the digital human's sense of reality and emotional expression ability;

[0035] (5) The present invention provides a method for improving the fluency of digital human facial expressions and movements. By improving the algorithm, the computational complexity is reduced, the processing speed is increased, and the efficiency and stability of the algorithm in actual deployment are ensured. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a method for improving the fluency of digital human facial expression movements in Embodiment 1 of the present invention;

[0037] Figure 2 is a schematic diagram of a triangular convolution kernel provided in Example 2 of the present invention;

[0038] Figure 3 It is the input data processed by the convolutional network in Example 3 of the present invention. DETAILED DESCRIPTION

[0039] The terms "first", "second", "third", "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices. The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments.

[0040] Example 1

[0041] According to a first aspect of the present invention, the present invention provides a method for improving the fluency of digital human facial expression movements. Figure 1 FIG. 1 shows a schematic flow chart of an embodiment of the present invention, as shown in FIG. Figure 1 As shown, the method comprises the following steps:

[0042] Step 1: Get the audio sent to the digital person and the text corresponding to the audio.

[0043] Specifically, the GLM4 large language model is used to generate audio with emotional coloring through speech synthesis engines such as VIT2; wherein the audio is single-channel WAV format data with a sampling rate of 8k.

[0044] Step 2: Process the audio data, extract the MFCC feature values ​​corresponding to the audio data, perform emotion recognition on the text, and obtain the text emotion.

[0045] In this embodiment, the MFCC value corresponding to the audio is calculated by the python dependent library librosa, specifically including:

[0046] Step 21: Use the audio processing library librosa to read the audio file and obtain the audio signal and sampling rate;

[0047] Step 22: Pre-emphasize the audio signal to reduce energy leakage in the low-frequency part;

[0048] Step 23: performing frame processing on the audio signal according to the frame length and the frame shift;

[0049] Step 24: For each frame of signal, first apply the window function to perform windowing, and then perform FFT transformation;

[0050] Step 25: Calculate the power spectrum after FFT transformation;

[0051] Step 26: Generate a Mel filter bank according to a given frequency range and number of Mel filter banks;

[0052] Step 27: Pass the power spectrum through the Mel filter bank to obtain the Mel spectrum;

[0053] Step 28: Perform logarithmic operation on the Mel spectrum and then perform discrete cosine transform to obtain MFCC coefficients;

[0054] Step 29: Apply a lifting window to the MFCC coefficients to improve recognition performance and return the MFCC feature matrix.

[0055] In addition, in this embodiment, the RoBERTa model is used to perform text emotion recognition on the language text corresponding to the audio to obtain the text emotion corresponding to the audio, wherein the text emotion includes six basic emotions, namely anger, disgust, fear, happiness, sadness and surprise;

[0056] The RoBERTa model preprocessing steps are as follows:

[0057] Text cleaning: First, clean the original text to remove useless punctuation, special characters, HTML tags, etc.

[0058] Word segmentation: Use the Byte-Level BPE word segmentation method to convert text into a token sequence that the model can understand;

[0059] Padding and truncation: To meet the input requirements of the model, truncate the overlong text and pad the shorter text;

[0060] Dynamic Masking: During the training process, dynamic masking technology is used to regenerate the mask at the beginning of each epoch to improve the generalization ability of the model.

[0061] The steps of RoBERTa model training are as follows:

[0062] Data preparation: Prepare annotated emotion recognition datasets, including texts with different emotion labels;

[0063] Model initialization: Use the pre-trained RoBERTa model for initialization, which can greatly speed up training and improve model performance;

[0064] Training: The preprocessed text data is input into the RoBERTa model, the loss function is calculated through forward propagation, and then the model parameters are updated using the backpropagation algorithm.

[0065] During the training process, the initial learning rate is 0.00005; the optimizer selects the Adam optimizer; and the BatchSize is 256.

[0066] Evaluation: After training, use the test set to evaluate the performance of the model, including accuracy, precision, recall, and F1 score.

[0067] The experimental data can be used for reference. The overall accuracy is 93%. The scores of various categories are shown in Table 1:

[0068] Table 1 Model performance evaluation table

[0069] Accuracy(%) Recall rate (%) F1 score (%) Positive emotions 90 92 91 Negative emotions 89 90 89.5 Neutral sentiment 87 85 86 Overall average 89 90 89.5

[0070] Step 3: Input the two-dimensional data composed of the MFCC feature value D1 and the text emotion into the pre-trained GR-TMCNN model to obtain the lip movements and facial movements of the digital human, wherein the GR-TMCNN model includes: a recurrent neural network and a convolutional network.

[0071] In this embodiment, the GR-TMCNN model is implemented by adding a convolutional layer on the basis of the gated recurrent unit GRU. The GRU layer first processes the input sequence to capture temporal features and long-term dependencies, and the convolutional layer extracts local features to better capture local patterns in the temporal data.

[0072] Step 3 specifically includes:

[0073] Step 31: Slice the feature value according to the time frame, and divide the MFCC feature value D1 into s MFCC segmentation values, that is, s tensor slices, where the shape of the first MFCC segmentation value is [80, 20], and the shapes of other MFCC segmentation values ​​are [160, 20], because the first lip shape state cannot observe its previous state.

[0074] Step 32: Input the tensor slice into the convolutional network to extract the MFCC temporal feature vector; use the MFCC temporal feature vector as input data through a 2-layer unidirectional recurrent neural network structure to obtain the lip shape result corresponding to the tensor slice.

[0075] Step 33: According to the order of tensor slices, the lip shape results corresponding to each slice are combined into the lip shape movement of the digital human; and the two-dimensional data composed of the text emotion and the lip shape movement is input into the recurrent neural network in the form of a 10ms data stream;

[0076] Step 34: For every 10ms of data, the recurrent neural network outputs the confidence of the facial muscle movements corresponding to each facial expression; selects the three facial expressions with the highest confidence, and weights the facial muscle movements corresponding to each facial expression according to the weights of the three facial expressions to obtain the facial movement corresponding to the data.

[0077] It should be noted that the lip shape results mentioned in this embodiment belong to a lip shape set, and the lip shape set includes: sil, PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou.

[0078] At the same time, the facial expressions mentioned in this embodiment belong to a preset expression set, which includes 21 facial expressions, 6 of which are the most basic expressions in Ekman's study in 1972: happiness, sadness, surprise, anger, disgust, and fear. The rest are 15 compound expressions discovered by Ohio State University in the United States in 2014 based on the 6 basic expressions. Each facial expression has corresponding facial muscle movements.

[0079] Step 4: Send the digital human's lip movements and facial movements to the digital human, thereby driving the digital human;

[0080] The obtained lip movements and facial movements are sent to the UE engine via data stream to drive the digital human, so that the facial expressions and lip movements of the digital human when speaking can be more consistent with human cognition.

[0081] Example 2

[0082] Based on Example 1, this example provides a triangle mask convolution used by the convolution network in the GR-TMCNN model.

[0083] Specifically, Figure 2 As shown, in this embodiment, the convolution kernel structure is triangular, and triangular convolution is introduced in the field of audio algorithms, which suppresses the influence of the traditional convolution kernel on subsequent data extraction during the convolution process. This convolution kernel is longer in the vertical direction, aiming to better capture the temporal characteristics of audio data, while being reduced in the horizontal direction to control the amount of calculation. At the same time, the convolution of the previous frame data is enhanced, while the convolution of the feature value data of the next frame is reduced, so that the model can learn more knowledge in the previous frame data and minimize the influence of the next frame data on the current frame while maintaining association with the next frame data.

[0084] In this embodiment, the convolution network also includes a post-random mask algorithm. Before each convolution operation, it is randomly determined whether to perform mask processing on the rightmost A9 element of the convolution kernel. If the mask is determined, its value is set to 0 to ignore its contribution in the convolution calculation, thereby improving the generalization of the model.

[0085] The triangular mask convolution method in this embodiment can better extract the data above, below and on the left side of the convolution point. At the same time, by randomly masking the data on the right side, it can control how to retain some "future" features and how many features to retain.

[0086] Example 3

[0087] Based on Example 2, this example provides specific convolution rules for triangular mask convolution.

[0088] Specifically, the triangle mask convolution process is as follows: Figure 3 For the input data shown, during the convolution operation, the center of the convolution kernel is spatially aligned with the currently processed convolution point. The convolution kernel slides on the input data in a preset manner. When the center position of the convolution kernel corresponds to point D11 in the input data, the convolution result is composed of the sum of the products of D11 and A7 in the convolution kernel, D21 and A8, and D12 and A9. The convolution result is:

[0089] D11*A7 + D21*A8 + mask*D12*A9

[0090] When the convolution point is D12, the convolution result is:

[0091] D11*A3 + D12*A7 + mask*D13*A9 + D21*A4 + D22*A8 + D31*A5

[0092] In the formula, if it is randomly decided to mask A9, the mask value is 0; if it is randomly decided not to mask A9, the mask value is 1.

[0093] As shown above, when the convolution operation is performed on the edge points in the image, only some of the nearby points are calculated, ensuring that the data points related to the time series are calculated without missing the eigenvalue attributes, and the MFCC features related to the time series are extracted, and the shape remains consistent before the convolution.

[0094] Example 4

[0095] According to the second aspect of the present invention, this embodiment provides an intelligent device, including a processor and a memory connected to the processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the steps of the method for improving the fluency of digital human facial expression movements described in any one of Examples 1-3 are executed.

[0096] Example 5

[0097] According to a third aspect of the present invention, this embodiment provides a computer storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for improving the fluency of digital human facial expression movements described in any one of embodiments 1-3 are implemented. The storage medium may include, for example, a storage component of a tablet computer, a hard disk of a computer, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a portable compact disk read-only memory (CD-ROM), a USB memory, or any combination of the above storage media. The computer storage medium may be any combination of one or more computer storage media.

[0098] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for improving the fluency of digital human facial expressions and movements, characterized in that: include: Step 1: Obtain the audio data sent to the digital human and the text corresponding to the audio data; Step 2: extracting the Mel-frequency cepstral coefficient MFCC feature value of the audio data, performing text emotion recognition on the text, and obtaining the text emotion; Step 3: Input the MFCC feature values ​​and the text emotions into the pre-trained GR-TMCNN model to obtain the lip movements and facial movements of the digital human; Step 4: Send the digital human's lip movements and facial movements to the digital human, thereby driving the digital human; Wherein, the GR-TMCNN model includes: a recurrent neural network and a convolutional network; The step 3 comprises: Divide the MFCC feature value into multiple tensor slices according to the time frame; Input tensor slices into the convolutional network to extract MFCC temporal feature vectors; Input the MFCC temporal feature vector into a recurrent neural network to obtain a corresponding lip shape result; According to the order of tensor slices, the lip shape results corresponding to each slice are combined into the lip shape movement of the digital human; The two-dimensional data consisting of the text emotion and lip shape movement is input into a recurrent neural network to obtain the facial movement of the digital human.

2. The method for improving the fluency of digital human facial expressions and movements according to claim 1, characterized in that: The two-dimensional data consisting of the text emotion and the lip shape movement is input into the recurrent neural network to obtain the facial movement of the digital human, including: Inputting the two-dimensional data consisting of the text emotion and lip shape movement into a recurrent neural network in the form of a preset time length data stream; For any preset duration data stream, the recurrent neural network outputs the confidence of the facial muscle movements corresponding to each facial expression; selects the n facial expressions with the highest confidence, and weights the facial muscle movements corresponding to each facial expression according to the weights of the n facial expressions to obtain the facial movement corresponding to the data, where n is a preset value.

3. The method for improving the fluency of digital human facial expressions and movements according to claim 2, characterized in that: The facial expression includes any expression in a preset expression set, and each facial expression has a corresponding facial muscle movement.

4. The method for improving the fluency of digital human facial expressions and movements according to claim 1, characterized in that: The convolution network includes: a triangular convolution, wherein the length of the convolution kernel of the triangular convolution in the vertical direction is greater than the length in the horizontal direction.

5. The method for improving the fluency of digital human facial expressions and movements according to claim 4, characterized in that: The convolution rule of the triangular convolution is: in the convolution operation, the triangular convolution kernel slides on the input data in a preset manner, and the convolution point that is spatially aligned with the center of the triangular convolution kernel is the current convolution point; in the convolution calculation, the product of the data of each point in the triangular convolution kernel and its corresponding convolution point is first calculated, and then all the products are added together to obtain the convolution result of the current convolution point.

6. The method for improving the fluency of digital human facial expressions and movements according to claim 5, characterized in that: The convolution network also includes: a post-mask convolution; the post-mask convolution is used to randomly decide whether to perform mask processing on the last element in the convolution kernel in the current convolution calculation before each convolution operation; in response to the decision to perform mask processing, the last element in the convolution kernel is recorded as 0.

7. The method for improving the fluency of digital human facial expressions and movements according to claim 1, characterized in that: The lip shape result includes any lip shape in a lip shape set, and the lip shape set includes: sil, PP, FF, TH, DD, kk, CH, SS, nn, RR, aa, E, ih, oh, ou.

8. The method for improving the fluency of digital human facial expressions and movements according to claim 1, characterized in that: The RoBERTa model is used to perform text emotion recognition on the text, and the text emotions include: anger, disgust, fear, happiness, sadness and surprise.

9. A smart device, characterized in that: include: a processor and a memory connected to said processor; A computer program is stored in the memory. When the computer program is executed by the processor, the steps of the method for improving the fluency of digital human facial expression movements as described in any one of claims 1 to 8 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for improving the fluency of digital human facial expression movements according to any one of claims 1 to 8 is implemented.

Citation Information

Cited By

  • Method for realizing synchronization of character expression and lip shape in video through emotion in sound and cloned digital human system

    CN120163906A

  • Method for realizing synchronization of character expression and lip shape in video with emotion in sound and cloned digital human system

    CN120163906B

  • Method and device for determining audio emotion and generating digital human video, and related product

    CN121483310A