Virtual digital human interaction method and device

Abstract

The invention discloses a virtual digital human interaction method and device. The method comprises the following steps: in response to received user input, processing the user input, generating a response text, and converting the response text into first voice content with universal voice features; acquiring voice features of a to-be-simulated target object, and converting the first voice content into second voice content with the voice features of the target object; and outputting the second voice content through the virtual digital human so as to interact with the user. According to the invention, the technical problem of inaccurate response during interaction with the virtual digital human is solved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and device for interacting with a virtual digital human. Background Art

[0002] With the rapid development of artificial intelligence technology, the application of virtual digital humans in customer service, education, entertainment and other fields has gradually increased. However, existing virtual digital human interaction technology often faces the problem of insufficient accuracy when responding to user input. This is mainly due to the limitations of speech synthesis technology, which makes the generated speech content lack personalization and realism, and the response text caused by insufficient input processing does not match the user's intention. Therefore, a new interaction method and device is urgently needed to enhance the interaction experience between virtual digital humans and users and ensure more accurate responses.

[0003] To address the above-mentioned problems, no effective solution has been proposed yet. Summary of the invention

[0004] The embodiments of the present invention provide a method and device for interacting with a virtual digital human, so as to at least solve the technical problem of inaccurate response when interacting with a virtual digital human.

[0005] According to one aspect of an embodiment of the present invention, there is provided an interaction method for a virtual digital human, comprising: in response to receiving user input, processing the user input, generating a response text, and converting the response text into a first voice content having a common voice feature; acquiring the voice feature of a target object to be simulated, converting the first voice content into a second voice content having the voice feature of the target object; and outputting the second voice content through the virtual digital human to interact with the user.

[0006] According to another aspect of an embodiment of the present invention, there is also provided an interaction method for a virtual digital human, comprising: a response generation module, configured to process the user input in response to receiving the user input, generate a response text, and convert the response text into a first voice content having a common voice feature; a conversion module, configured to obtain the voice features of a target object to be simulated, and convert the first voice content into a second voice content having the voice features of the target object; and an interaction module, configured to output the second voice content through the virtual digital human to interact with the user.

[0007] In an embodiment of the present invention, in response to receiving user input, the user input is processed to generate a response text, and the response text is converted into a first voice content with a general voice feature; the voice feature of the target object to be simulated is obtained, and the first voice content is converted into a second voice content with the voice feature of the target object; the second voice content is output through the virtual digital human to interact with the user. Through the above solution, the technical problem of inaccurate response when the user interacts with the virtual digital human in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0009] Figure 1 is a flow chart of a virtual digital human interaction method according to an embodiment of the present invention;

[0010] Figure 2 is a flow chart of another virtual digital human interaction method according to an embodiment of the present invention;

[0011] Figure 3 is a flow chart of a method for generating a response text according to an embodiment of the present invention;

[0012] Figure 4 is a flow chart of a method for acquiring speech features of a target object according to an embodiment of the present invention;

[0013] Figure 5 is a flow chart of a method for speech feature mapping and conversion according to an embodiment of the present invention;

[0014] Figure 6 is a flow chart of an action-driven method according to an embodiment of the present invention;

[0015] Figure 7 is a structural schematic diagram of an interactive device for a virtual digital human according to an embodiment of the present invention;

[0016] Figure 8 A schematic diagram of the structure of an electronic device suitable for implementing the embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0018] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0019] According to an embodiment of the present invention, a method embodiment of a virtual digital human interaction method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0020] Figure 1 is an interactive method of a virtual digital human according to an embodiment of the present invention, such as Figure 1 As shown, the method comprises the following steps:

[0021] Step S102, in response to receiving the user input, processing the user input, generating a response text, and converting the response text into a first voice content having a general voice feature.

[0022] In response to receiving user input, the system first processes the user input. User input usually includes text, voice or other forms of data. Virtual digital human is a computer-generated character used to simulate human behavior and communication, and is widely used in multiple scenarios such as virtual reality and human-computer interaction. Virtual digital human generates corresponding feedback or performs corresponding operations based on user input. Specifically, the system analyzes and processes the data input by the user and generates response text corresponding to the input content.

[0023] The generated response text is then converted into a first speech content with universal speech features. Universal speech features refer to universal characteristics used to characterize speech, such as pitch, speaking speed, intonation, etc. These features can be applied to a variety of different speech models to ensure that the generated speech content is universal and natural. The speech conversion process relies on speech synthesis technology to first generate basic speech output. The universal speech content can be further converted into personalized speech features for a specific target object. The target object refers to a specific person or role whose speech features need to be simulated. It can be a real person or a fictional character.

[0024] Step S104: acquiring speech features of a target object to be simulated, and converting the first speech content into second speech content having the speech features of the target object.

[0025] First, obtain the speech features of the target object to be simulated. For example, obtain the audio and video clip of the target object, segment the audio and video clip according to a preset time length, and obtain multiple video sub-segments; extract the speech features of the target object including fundamental frequency, resonance peak and timbre features by analyzing the spectral features of the speech content in each video sub-segment. By obtaining the audio and video clip of the target object and segmenting it into multiple video sub-segments, the present application can analyze the speech spectral features of each sub-segment in detail, extract speech features such as fundamental frequency, resonance peak and timbre, thereby improving the accuracy and efficiency of speech feature extraction, making the generated speech synthesis content more personalized and natural, improving the speech expressiveness of the virtual digital human, and thus enhancing the user experience and the realism of virtual interaction.

[0026] Next, the first voice content is converted into a second voice content having the voice features of the target object. Specifically, the universal voice features are extracted from the first voice content, and the universal voice features are mapped using a deep learning model based on the voice features of the target object. For example, the voice features of the target object are input into the deep learning model; the deep learning model maps the universal voice features by minimizing the difference between the voice features of the target object and the universal voice features. By extracting universal voice features and minimizing the difference between them and the voice features of the target object, the deep learning model can accurately adjust the network parameters to achieve feature mapping, thereby ensuring that the generated second voice content more realistically simulates the unique voice features of the target object and enhances the personalization and naturalness of the virtual digital human voice output.

[0027] Then, based on the voice features of the mapped target object, the deep learning model is used to convert the first voice content into the second voice content.

[0028] Step S106: output the second voice content through the virtual digital human to interact with the user.

[0029] First, collect interjections and the corresponding motion capture data of the target object from the audio-visual clip, and establish a mapping relationship between the collected interjections and the motion capture data, wherein the motion capture data includes facial movements and / or body movements of the target object; and construct an action library of the virtual digital human based on the mapping relationship.

[0030] Next, modal particles are extracted from the response text, and action information is extracted from a pre-built action library of the virtual digital human based on the extracted modal particles; based on the extracted action information, the action of the virtual digital human is driven, wherein the action of the virtual digital human includes facial action and body action. While driving the body and facial expression actions of the virtual digital human, the second voice content is also outputted through the virtual digital human.

[0031] Figure 2 Another method for interacting with a virtual digital person according to an embodiment of the present invention is as follows. Figure 2 As shown, the method mainly includes the following steps:

[0032] Step S202: user input processing and response text generation.

[0033] The system first responds to the user's input. User input can include text, voice, gesture, image or other forms of data. Regardless of the form of input, the system uses natural language processing (NLP) technology to analyze the user's input content, extract key information, and generate corresponding response text.

[0034] like Figure 3 As shown, the method for generating a response text includes the following steps:

[0035] Step S2022, input recognition.

[0036] The system first recognizes and classifies user input through speech recognition technology or text parsing technology. For speech input, a combination of acoustic model and language model is used to ensure the accuracy of recognition. For text input, grammar analysis, sentiment analysis and other technologies are used to extract user intentions and emotional states. For image input, the image needs to be preprocessed, then image recognition is performed, and the results of image recognition are converted into text.

[0037] This embodiment takes image input as an example for detailed description. First, generate a Gaussian kernel and normalize the Gaussian kernel. Generate a Gaussian kernel and perform normalization. Assume that the image has been loaded as a matrix I, where I(i,j) represents the pixel value of the image at position (i,j). First, select the size k (such as 3x3, 5x5, 7x7 and other positive odd numbers) and standard deviation σ of the Gaussian kernel. A larger Gaussian kernel will increase the blurring effect, but may lose more details; the larger the standard deviation σ, the more significant the blurring effect. Next, generate a Gaussian kernel matrix K based on the Gaussian function.

[0038]

[0039] Among them, σ is the standard deviation, which determines the width of the Gaussian distribution, (x, y) is the coordinate of the pixel in the picture relative to the center of the kernel, and k is the size of the Gaussian kernel. The Gaussian kernel generated in this embodiment is normalized to ensure that when the image is blurred, the effects of different kernels on the pixel values ​​remain consistent, avoiding unnecessary deviations. By selecting the appropriate Gaussian kernel size k and standard deviation σ, users can flexibly adjust the blur effect according to specific needs. A larger Gaussian kernel can enhance the degree of blur, and an increase in the standard deviation σ further enhances the blur effect, allowing users to achieve a better balance between noise reduction and feature highlighting. In addition, by applying the Gaussian kernel matrix K generated by the Gaussian function, the image can be effectively smoothed, noise can be reduced, and the main features can be enhanced, significantly improving the overall quality of the image. The method provided in this embodiment has a wide range of application potentials in the field of image processing, including noise reduction, edge detection, and image enhancement, which helps to improve the visual effect and clarity of the final image.

[0040] For a k×k Gaussian kernel, whose center is (k / / 2, k / / 2), the Gaussian kernel is normalized to ensure that its weighted sum is 1:

[0041]

[0042] Wherein, m represents the offset of the currently processed pixel position in the horizontal direction. n represents the offset of the currently processed pixel position in the vertical direction. The normalization of this embodiment ensures that the sum of pixel values ​​remains consistent when the Gaussian kernel is applied to the image, so that the blurring effect is smooth and natural. In addition, by defining m and n to represent the offset of the currently processed pixel position in the horizontal and vertical directions, the user can accurately control the influence of each pixel in the blurring process, further enhancing the flexibility and accuracy of image processing. The method provided in the present application has significant beneficial effects in image denoising and feature smoothing, and improves the overall quality of image processing.

[0043] Then, denoising is performed. Multi-scale Gaussian blur captures the features of the image at different scales by applying multiple Gaussian kernels with different standard deviations σ\sigmaσ, thereby enhancing the denoising effect. The improved denoising formula is:

[0044]

[0045] Among them, I(i,j) is the pixel value of the original image at position (i,j), m,n are the offset of the horizontal coordinate position and the vertical coordinate position relative to the center of the Gaussian kernel. In the improved denoising processing formula, the pixel value I(i,j) of the original image and the horizontal and vertical coordinate offsets m and n relative to the center of the Gaussian kernel work together to ensure the meticulousness of the denoising process. tx is the horizontal gradient of the image at each pixel position, ty is the vertical gradient of the image at each pixel position, and β is the gradient weight. By introducing the horizontal gradient tx and vertical gradient ty at each pixel position of the image and setting the gradient weight β, the sensitivity in the denoising process can be adjusted more flexibly to adapt to different image characteristics. N is the noise intensity parameter, which is used to control the sensitivity to the difference in pixel values ​​of the image during the denoising process. It is obtained based on the noise intensity η, the pixel value of the image at position (i,j) and the offset value of the pixel value of the image at position (i,j). The introduction of the noise intensity parameter N further enhances the sensitivity to the difference in pixel values ​​based on the noise intensity η and the offset of pixel values, thereby optimizing the denoising effect. This embodiment can effectively distinguish noise from image details by combining blur processing at different scales, thereby improving the accuracy of denoising.

[0046] in,

[0047] Afterwards, the preprocessed image is recognized using an image recognition model to obtain a recognition result in text format. The image recognition model is a prior art, and therefore, will not be described in detail here.

[0048] Step S2024, semantic understanding.

[0049] After the user input is recognized or converted into text, the system needs to perform semantic understanding. Using deep learning models, such as Transformer model-based architectures (such as BERT, GPT, etc.), the context of the input is analyzed and the user's true intention is understood.

[0050] Step S2026, generate response text.

[0051] After semantic understanding, the system uses a deep learning model to generate response text.

[0052] Step S204: Generate first voice content.

[0053] The generated response text is then converted into the first voice content with common voice features. Common voice features refer to common attributes used to describe voice, such as pitch, speed, timbre and other features. For example, the end-to-end text-to-speech (TTS) model based on deep learning converts text input into voice output, and makes the generated voice natural and fluent by learning a large amount of voice data. In the process of generating the first voice content, the system will first extract the common voice features of the input text. These features are crucial for subsequent personalized voice generation and can provide a basis for subsequent voice feature mapping.

[0054] Step S206, obtaining the speech features of the target object.

[0055] In this step, the system obtains the speech features of the target object to be simulated, so as to convert the first speech content into the second speech content having the features of the target object.

[0056] like Figure 4 As shown, the method comprises the following steps:

[0057] Step S2062: target object selection.

[0058] The system first needs to obtain the target object. The target object can be a public figure in reality, a film or TV character, etc. After selecting the target object, the system will start collecting the voice data of the target object.

[0059] Step S2064, voice data collection.

[0060] By acquiring the audio and video clips of the target object, the system will segment them and generate multiple video sub-segments. The collected audio and video content can come from public speeches, film and television works, social media, etc. The system can automatically process these clips according to the preset time length to ensure the diversity and representativeness of the sample.

[0061] Step S2066, speech feature extraction.

[0062] After analyzing each video sub-clip, the system will extract the target object's speech features, including fundamental frequency, formant, timbre, etc. These spectral features are used to subsequently generate personalized speech output, making the final speech content closer to the target object's real voice.

[0063] Step S208: speech feature mapping and conversion.

[0064] After acquiring the voice features of the target object, the system converts the first voice content into second voice content having the features of the target object.

[0065] like Figure 5 As shown, the method comprises the following steps:

[0066] Step S2082, build and train a deep learning model.

[0067] Use deep learning technology to build a neural network model to map common speech features to the speech features of the target object. First, select a network architecture, such as a convolutional neural network (CNN), a recurrent neural network (RNN), or a long short-term memory network (LSTM), which can effectively process time series data and audio signals. Second, select an activation function (such as ReLU, sigmoid, or tanh) and a loss function (such as mean square error loss or cross entropy loss). In addition, configure an optimizer (such as Adam or SGD) to improve the efficiency and effectiveness of model training.

[0068] First, the target object's speech is input into the deep learning model to perform forward propagation. The specific steps are as follows: the processed target object speech feature vector is input into the input layer of the network, and the model performs feature conversion through hidden layers (such as fully connected layers, convolutional layers, or recurrent layers). At each layer, the activation value of the node is calculated, and the formula is:

[0069] z (l) =W (l) a (l-1) +b (l)

[0070] Among them, z (l) is the linear combination of the lth layer, W (l) and b (l) are the weight matrix and bias vector of the lth layer, respectively, and a (l -1) is the output of the previous layer. z is activated by an activation function (such as ReLU, Sigmoid, etc.) (l) Perform nonlinear transformation to get the output of the current layer:

[0071] a (l) =activation(z (l) )

[0072] The output of the last layer will be used as the predicted speech features of the target object. The system compares the generated output with the speech features of the target object and calculates the loss function to evaluate the performance of the model:

[0073] L = Loss (a (L) ,y)

[0074] Among them, a (L) is the predicted value of the output layer, and y is the actual speech feature of the target object.

[0075] After calculating the loss, the back-propagation algorithm is used to adjust the network parameters to minimize the loss. The specific steps include: First, the loss function L is derived to calculate the gradient of each layer's weights and biases. Using the chain rule, the gradient of each layer is calculated by propagating from the output layer to the input layer in turn:

[0076]

[0077] in, is the gradient of the loss function to the output of the current layer, ⊙ represents element-by-element multiplication, activation′(z (l) ) is the derivative of the activation function. Use gradient descent to update the weights and biases of each layer:

[0078]

[0079] Among them, α is the learning rate, which controls the update step. The above forward propagation and back propagation process will be iterated continuously until the performance of the model on the validation set is no longer significantly improved.

[0080] During training, the system can use the following regularization techniques to avoid overfitting: Add a regularization term to the loss function, which is the sum of the squares of all weights to control model complexity:

[0081]

[0082] Among them, λ is the regularization strength hyperparameter. A certain proportion of neurons are randomly discarded in each training batch, so that the model does not depend on specific neurons and enhances the generalization ability of the model.

[0083] Through the above steps, the general voice features can be effectively mapped to the voice features of the target object, thereby generating personalized and natural voice content and improving the user's interactive experience.

[0084] After training the model, in some embodiments, the model can also be quantized to avoid wasting resources. The quantized data is calculated based on the minimum and maximum values ​​of the original data and the number of quantization bits; the number of quantization bits is dynamically updated according to the variance, entropy value and balance coefficient of the quantized data to obtain the optimal number of quantization bits; then, new quantized data is regenerated based on this optimal number of quantization bits. The specific quantization formula is as follows:

[0085]

[0086] Among them, Q old(x,b) is the data before quantization, Q(x) is the data after quantization, x is the original data, b is the number of quantized bits, and min(x) and max(x) are the minimum and maximum values ​​of the original data, respectively. opt Obtained by the following formula:

[0087] bopt=OptimizeBits(Var(Q(x)),H(Q(x)),λ)

[0088] Among them, b opt is the optimal number of quantization bits, Var(Q(x)) is the variance of the quantized data, H(Q(x)) is the entropy of the quantized data, and λ is the balance factor.

[0089] Step S2084, feature mapping.

[0090] In the feature mapping stage, the system inputs the common speech features into the trained deep learning model. The model starts to perform forward propagation, calculates the difference between the target object's speech features and the common speech features, and outputs the mapped speech features of the target object until the difference between the two is minimized.

[0091] Step S2086, generating second voice content.

[0092] After the feature mapping is completed, the second voice content is generated based on the voice features of the mapped target object. The generated voice content can be implemented in a variety of ways, such as using a vocoder (such as WaveNet or Griffin-Lim algorithm) to convert the output of the deep learning model into a final audio signal. During the generation process, the pitch, speech rate, volume and other parameters of the generated voice content can be adjusted to further enhance the natural fluency of the voice. Through the above steps, the constructed deep learning model can effectively realize the accurate mapping between the general voice features and the voice features of the target object, and then generate personalized and natural voice content to meet the needs of virtual digital human interaction.

[0093] Step S210, action driving.

[0094] The interaction of virtual digital humans not only relies on voice output, but also needs to be combined with corresponding actions and expressions to enhance the realism of the interaction.

[0095] like Figure 6 As shown, the action-driven method includes the following steps:

[0096] Step S2102, constructing an action library.

[0097] Extract facial and body movement data from the target object's video and audio clips, and use motion capture technology to establish a mapping relationship between motion capture data and modal particles. Motion capture data can be captured by special equipment and software, such as 3D motion capture cameras. Based on the extracted motion capture data, establish a motion library of virtual digital humans. The motion library contains a variety of movements and expressions, which can provide appropriate feedback in different situations.

[0098] Specifically, the collected audio and video clips are processed, and the modal particles are extracted using audio processing technology (such as audio segmentation, speech recognition, etc.). Modal particles refer to words used to express the speaker's emotions, attitudes or reactions in conversation, such as "um", "ah", "oh", etc. The system will identify and record these modal particles by analyzing audio signals. In the same time period, the facial movements and body movements of the target object are recorded using motion capture technology. These motion capture data should include: facial movements, such as smiling, frowning, blinking, etc., used to convey emotional changes; body movements, such as gestures, body tilts, posture changes, etc., used to enhance the expression effect of modal particles. After obtaining the modal particles and the corresponding motion capture data, the system needs to establish a mapping relationship between the two. Ensure that the modal particles and their corresponding motion capture data match each other in time so as to establish an accurate time correspondence. Then, the extracted motion capture data is feature analyzed to identify which facial or body movements are associated with specific modal particles. Based on the established mapping relationship, a motion library of virtual digital humans is constructed. The action library contains the association between modal particles and actions. Each modal particle is connected to a set of corresponding facial and body movements to form a specific expression pattern. At the same time, the action library is classified according to emotion, scene or context for quick call in subsequent applications. Through the rich action library, the virtual digital human can express emotions more vividly and realistically when interacting with users, enhancing the naturalness and effectiveness of human-computer interaction.

[0099] Step S2104, dynamic response.

[0100] The modal particles are extracted from the response text, and the corresponding action information is obtained from the action library based on the extracted modal particles. The system will drive the actions of the virtual digital human, such as facial expressions and body movements. At the same time, in the process of driving the action, the virtual digital human also outputs the second voice content to achieve a high degree of coordination between voice and action.

[0101] The present application also provides an interactive device for a virtual digital human, such as Figure 7As shown, it includes: a response generation module 72, which is configured to process the user input in response to receiving the user input, generate a response text, and convert the response text into a first voice content with a common voice feature; a conversion module 74, which is configured to obtain the voice feature of the target object to be simulated, and convert the first voice content into a second voice content with the voice feature of the target object; an interaction module 76, which is configured to output the second voice content through the virtual digital human to interact with the user.

[0102] It should be noted that the virtual digital human interaction device provided in the above embodiment is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the virtual digital human interaction device provided in the above embodiment and the virtual digital human interaction method embodiment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0103] Figure 8 FIG. 1 shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiment of the present disclosure. It should be noted that: Figure 8 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0104] like Figure 8 As shown, the electronic device includes a central processing unit (CPU) 1001, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage part 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for system operation are also stored. The CPU 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0105] The following components are connected to the I / O interface 1005: an input section 1006 including a keyboard, a mouse, etc.; an output section 1007 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1008 including a hard disk, etc.; and a communication section 1009 including a network interface card such as a LAN card, a modem, etc. The communication section 1009 performs communication processing via a network such as the Internet. A drive 1010 is also connected to the I / O interface 1005 as needed. A removable medium 1011, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1010 as needed, so that a computer program read therefrom is installed into the storage section 1008 as needed.

[0106] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A virtual digital human interaction method, characterized in that: include: In response to receiving user input, processing the user input, generating a response text, and converting the response text into a first voice content having a general voice feature; Acquire speech features of a target object to be simulated, and convert the first speech content into second speech content having the speech features of the target object; The second voice content is outputted through the virtual digital human to interact with the user.

2. The method according to claim 1, characterized in that Obtain the speech features of the target object to be simulated, including: Acquire the video and audio segments of the target object, and segment the video and audio segments according to a preset time length to obtain a plurality of video sub-segments; By analyzing the frequency spectrum features of the speech content in each video sub-segment, speech features of the target object including fundamental frequency, formant and timbre features are extracted.

3. The method according to claim 1, characterized in that: Converting the first speech content into a second speech content having the speech features of the target object includes: Extracting the common speech features from the first speech content, and mapping the common speech features using a deep learning model based on the speech features of the target object; Based on the voice features of the mapped target object, the deep learning model is used to convert the first voice content into the second voice content.

4. The method according to claim 3, characterized in that Mapping the general speech feature using a deep learning model based on the speech feature of the target object includes: Inputting the speech features of the target object into the deep learning model; The deep learning model maps the common speech features by minimizing the difference between the speech features of the target object and the common speech features.

5. The method according to claim 2, characterized in that: While the second voice content is outputted by the virtual digital human, the method further includes: Extracting modal particles from the response text, and extracting action information from a pre-built action library of the virtual digital human based on the extracted modal particles; Based on the extracted action information, the action of the virtual digital human is driven, wherein the action of the virtual digital human includes facial action and body action.

6. The method according to claim 4, characterized in that Before extracting action information from a pre-built action library of the virtual digital human based on the extracted modal particles, the method further includes: Collecting interjections and corresponding motion capture data of the target object from the video and audio clips, and establishing a mapping relationship between the collected interjections and the motion capture data, wherein the motion capture data includes facial movements and / or body movements of the target object; Based on the mapping relationship, an action library of the virtual digital human is constructed.

7. An interactive device for a virtual digital human, characterized in that: include: A response generation module, configured to, in response to receiving a user input, process the user input, generate a response text, and convert the response text into a first voice content having a general voice feature; A conversion module, configured to obtain the speech characteristics of the target object to be simulated, and convert the first speech content into a second speech content having the speech characteristics of the target object; The interaction module is configured to output the second voice content through the virtual digital human to interact with the user.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 6.

9. A computer device, characterized in that: include: Memory and processor, The memory stores a computer program; The processor is used to execute the computer program stored in the memory, and when the computer program is run, the processor is enabled to execute the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images