Method for generating virtual digital human, electronic equipment and storage medium

Abstract

The invention discloses a method for generating a virtual digital human, electronic equipment and a storage medium, and belongs to the field of Internet. The method comprises the following steps: acquiring feature information associated with input voice; obtaining a target result associated with the virtual digital human based on the feature information; based on the target result and an image quantification model, obtaining an image feature recommendation result of the virtual digital human; based on the target result and an expression quantification model, obtaining an expression feature recommendation result of the virtual digital human; based on the target result and an action quantification model, obtaining an action feature recommendation result of the virtual digital human; based on the image feature recommendation result of the virtual digital person, the expression feature recommendation result of the virtual digital person and the action feature recommendation result of the virtual digital person, an element co-processing result is obtained, and the element co-processing result is used for generating the virtual digital person.

Description

Technical Field

[0001] The present application belongs to the field of Internet, and specifically relates to a method for generating a virtual digital human, an electronic device and a storage medium. Background Art

[0002] With the development of technology, virtual digital people came into being in order to better meet users' personalized services and reduce the operating costs of manual customer service.

[0003] In the process of generating virtual digital humans, related technologies often only consider making the mouth shape of the virtual digital humans change accordingly with the changes in speech. As a result, since the generated virtual digital humans are not natural and realistic enough, users will feel unreal when interacting with the virtual digital humans, which affects their acceptance. Summary of the invention

[0004] The embodiments of the present application provide a method for generating a virtual digital human, an electronic device, and a storage medium, which can solve the problem that the virtual digital human generated by the related technology is not natural and realistic enough.

[0005] In a first aspect, an embodiment of the present application provides a method for generating a virtual digital human, comprising: Acquiring feature information associated with the input speech; Based on the feature information, a target result associated with the virtual digital human is obtained; Based on the target result and the image quantification model, obtaining the image feature recommendation result of the virtual digital human; Based on the target result and the expression quantification model, obtaining the expression feature recommendation result of the virtual digital human; Based on the target result and the action quantification model, obtaining the action feature recommendation result of the virtual digital human; Based on the recommendation results of the image features of the virtual digital human, the recommendation results of the expression features of the virtual digital human and the recommendation results of the action features of the virtual digital human, an element collaborative processing result is obtained, and the element collaborative processing result is used to generate a virtual digital human.

[0006] In a second aspect, an embodiment of the present application provides a device for generating a virtual digital human, including: An acquisition module, used to acquire feature information associated with the input speech; The processing module is used to obtain a target result associated with the virtual digital human based on the feature information; obtain a recommendation result of the image feature of the virtual digital human based on the target result and the image quantification model; obtain a recommendation result of the expression feature of the virtual digital human based on the target result and the expression quantification model; obtain a recommendation result of the action feature of the virtual digital human based on the target result and the action quantification model; obtain an element coordination processing result based on the recommendation result of the image feature of the virtual digital human, the recommendation result of the expression feature of the virtual digital human and the recommendation result of the action feature of the virtual digital human, and the element coordination processing result is used to generate the virtual digital human.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, which includes a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the method described in the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed, the steps of the method described in the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it implements the steps of the method described in the first aspect, the second aspect or the third aspect.

[0010] In the embodiment of the present application, feature information associated with the input voice is obtained; based on the feature information, a target result associated with the virtual digital human is obtained; based on the target result and the image quantification model, an image feature recommendation result of the virtual digital human is obtained; based on the target result and the expression quantification model, an expression feature recommendation result of the virtual digital human is obtained; based on the target result and the action quantification model, an action feature recommendation result of the virtual digital human is obtained; based on the image feature recommendation result of the virtual digital human, the expression feature recommendation result of the virtual digital human, and the action feature recommendation result of the virtual digital human, an element collaborative processing result is obtained, and the element collaborative processing result is used to generate the virtual digital human. In this way, in the process of generating the virtual digital human, by introducing the image quantification model, the expression quantification model and the action quantification model, the virtual digital human is quantified from multiple dimensions such as image, expression and action, and then the quantified results are collaboratively processed to obtain the element collaborative processing result, so that the generated virtual digital human integrates multiple simulation elements, which can be more natural and real compared with a single simulation element, solving the problem that the virtual digital human generated by the related technology is not natural and realistic enough. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0012] Figure 1 is a flow chart of a method for generating a virtual digital human provided by an embodiment of the present application; Figure 2 is a flowchart of another method for generating a virtual digital human provided by an embodiment of the present application; Figure 3 is a flowchart of another method for generating a virtual digital human provided by an embodiment of the present application; Figure 4 It is a structural block diagram of a device for generating a virtual digital human provided by an embodiment of the present application; Figure 5 It is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0013] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0014] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0015] With the continuous breakthroughs in artificial intelligence technology and computing power, virtual digital human image engine products will become more and more intelligent and anthropomorphic, and can handle more complex emotional problems and truly enter the hearts of users. They can gradually replace real people to provide services, significantly saving labor costs. On the other hand, as the public pays more and more attention to privacy protection, the use of virtual digital human image voice shaping and quantitative collaboration to achieve human-computer communication will better protect user privacy, make users trust and feel safe, and objectively have a broad user market. The method for generating virtual digital people provided in the embodiment of this application can be widely used in a variety of intelligent interactive products, such as virtual customer service, virtual shopping guide, virtual anchor, etc.

[0016] The method for generating a virtual digital person provided in the embodiment of the present application combines the sound shaping technology with the multi-element quantification of the virtual image to achieve refined interaction and generate emotional value, which can be applied to the telecom customer service scene. Specifically, the embodiment of the present application refines the manipulation of various elements of the virtual digital person so that the virtual digital person can express various gestures and actions, be more anthropomorphic and be able to express more emotional intentions. At the same time, attention is paid to the coordination and change relationship between the elements such as the action, voice, lips and background of the virtual digital person, and the interaction logic of each element is optimized to enable the user to experience a smooth and natural interactive experience. In addition, through the multi-element quantification algorithm, the virtual digital person's expression, action and other simulation elements are classified and quantified to form a proprietary feature library that can be used for the linkage manipulation of the virtual image. The finer the classification and quantification granularity, the richer the manipulation expression intention, and it can be based on this to create a common capability engine with richer emotions, more realistic and anthropomorphic virtual expressions. This technology will further enhance the personalized customization ability of the virtual digital person, and can provide users with personalized virtual digital human images according to user needs and scene characteristics. For example, users can choose different virtual digital human images, animation styles and voice characteristics according to their preferences. In summary, the embodiment of the present application combines the voice shaping technology and the virtual image multi-element quantification technology, and can provide more anthropomorphic and personalized virtual digital human services by optimizing the control and design of the virtual digital human.

[0017] Figure 1 : is a flow chart of a method for generating a virtual digital human provided by an embodiment of the present application. Figure 1 A method for generating a virtual digital human provided by an embodiment of the present application may include: Step 110, acquiring feature information associated with the input speech; The feature information may include basic voice features, voice change trend features, emotion features, and emotion change features.

[0018] The basic voice feature may include at least one of pitch, speed and volume. The voice change trend feature may indicate a change trend of the basic voice feature.

[0019] The voice change trend characteristics may include sudden increase (sudden increase), sudden decrease (sudden decrease) and fluctuation, etc. Thus, the voice change trend characteristics may include: sudden increase in pitch, sudden decrease in pitch, pitch fluctuation, sudden increase in sound speed, sudden decrease in sound speed, sound wave fluctuation, sudden increase in volume, sudden decrease in volume and volume fluctuation.

[0020] The emotional characteristics may include at least one of positive emotions, neutral emotions, and negative emotions. Each category of emotional characteristics may include multiple types. Positive emotions may include politeness, tolerance, etc., neutral emotions may include embarrassment, urgency, and help-seeking, etc., and negative emotions may include complaints, complaints, anxiety, and confrontation.

[0021] Emotional change characteristics can reflect the changes in emotions. Specifically, emotional change characteristics may include: deepening of negative emotions, maintenance of negative emotions, weakening of negative emotions, generation of negative emotions, deepening of positive emotions, maintenance of neutral emotions, abnormal fluctuations in emotions, etc.

[0022] In the embodiment of the present application, feature information associated with the input voice can be obtained by voice shaping. Voice shaping refers to converting the conveyed emotions, intentions, etc. into multiple elements such as expressions, actions, and backgrounds of the virtual image based on the results of qualitative analysis of voice features, and expressing them synchronously with customer service voice feedback to enhance user acceptance and reduce confrontation, so as to better solve problems. Compared with traditional virtual digital humans, the embodiment of the present application refines the manipulation of various elements of virtual digital humans, allowing virtual digital humans to express various postures and actions, making them more humanized and able to express more emotional intentions.

[0023] The input speech may be the speech of a user within a period of time. A speech with a time slice length of m milliseconds may be selected as the input speech, where m is a positive integer. Feature information of the input speech may be extracted as feature information associated with the input speech. In the process of obtaining the speech change trend feature, the basic feature change trend may be analyzed for adjacent time slices. After obtaining the basic speech feature and the speech change trend feature, the emotion feature may be obtained based on the basic speech feature and the speech change trend feature, and then the emotion change feature may be obtained.

[0024] Step 120, obtaining a target result associated with the virtual digital human based on the feature information; Step 130, obtaining a recommendation result of the image feature of the virtual digital human based on the target result and the image quantification model; Step 140, obtaining a recommendation result of the expression feature of the virtual digital human based on the target result and the expression quantification model; Step 150, obtaining a recommended result of the action feature of the virtual digital human based on the target result and the action quantification model; Step 160, based on the recommendation results of the image features of the virtual digital human, the recommendation results of the expression features of the virtual digital human, and the recommendation results of the action features of the virtual digital human, obtain the element collaborative processing results, and the element collaborative processing results are used to generate the virtual digital human.

[0025] The image feature includes the background feature, and the image feature recommendation result includes the background feature recommendation result.

[0026] In the embodiment of the present application, feature information associated with the input voice is obtained; based on the feature information, a target result associated with the virtual digital human is obtained; based on the target result and the image quantification model, an image feature recommendation result of the virtual digital human is obtained; based on the target result and the expression quantification model, an expression feature recommendation result of the virtual digital human is obtained; based on the target result and the action quantification model, an action feature recommendation result of the virtual digital human is obtained; based on the image feature recommendation result of the virtual digital human, the expression feature recommendation result of the virtual digital human, and the action feature recommendation result of the virtual digital human, an element collaborative processing result is obtained, and the element collaborative processing result is used to generate the virtual digital human. In this way, in the process of generating the virtual digital human, by introducing the image quantification model, the expression quantification model and the action quantification model, the virtual digital human is quantified from multiple dimensions such as image, expression and action, and then the quantified results are collaboratively processed to obtain the element collaborative processing result, so that the generated virtual digital human integrates multiple simulation elements, which can be more natural and real compared with a single simulation element, solving the problem that the virtual digital human generated by the related technology is not natural and realistic enough.

[0027] In an embodiment of the present application, there may be multiple implementations of obtaining a target result associated with a virtual digital human based on the feature information in step 120. Specifically, in one implementation, step 120 may include: inputting the feature information into a machine learning model to obtain a target result associated with a virtual digital human. The target result may include a virtual image drive vector set, and may also include both a virtual digital human service strategy and the virtual image drive vector set. In another implementation, step 120 may include: obtaining a virtual digital human service strategy based on the feature information; obtaining a virtual image drive vector set based on the feature information and the virtual digital human service strategy, the virtual image drive vector set acting on the virtual customer; and determining the virtual digital human service strategy and the virtual image drive vector set as the target result associated with the virtual digital human.

[0028] In an embodiment of the present application, obtaining a virtual image drive vector set based on the feature information and the virtual digital human service strategy may include: inputting the feature information and the virtual digital human service strategy into a machine learning model; obtaining a virtual image drive vector set through the machine learning model; wherein the virtual image drive vector set matches the feature information and the virtual digital human service strategy.

[0029] The virtual image driving vector set in the embodiment of the present application is used to apply the behavior that embodies the virtual digital human service strategy to the virtual digital human, for example, it can be used to drive the changes of various actions, expressions and images (including background) of the virtual digital human.

[0030] In an embodiment of the present application, obtaining the image feature recommendation result of the virtual digital person based on the target result and the image quantification model in step 130 may include: obtaining an image classification; inputting the target result and the image classification into the image quantification model to obtain the image feature recommendation result of the virtual digital person.

[0031] In an embodiment of the present application, obtaining the expression feature recommendation result of the virtual digital human based on the target result and the expression quantification model in step 140 may include: obtaining an expression classification; inputting the target result and the expression classification into the expression quantification model to obtain the expression feature recommendation result of the virtual digital human.

[0032] In an embodiment of the present application, obtaining the action feature recommendation result of the virtual digital human based on the target result and the action quantification model in step 150 may include: obtaining the action classification; inputting the target result and the action classification into the action quantification model to obtain the action feature recommendation result of the virtual digital human.

[0033] In an embodiment of the present application, the step 160 of obtaining the element coordination processing result based on the image feature recommendation result of the virtual digital person, the expression feature recommendation result of the virtual digital person, and the action feature recommendation result of the virtual digital person may include: inputting the image feature recommendation result of the virtual digital person, the expression feature recommendation result of the virtual digital person, and the action feature recommendation result of the virtual digital person into an element coordination model to obtain the element coordination processing result. The element coordination model may be a trained machine learning model, which may be trained based on image classification, expression classification, action classification, and the target result.

[0034] In an embodiment of the present application, the element collaborative processing result may include a virtual image state vector, and the virtual image state vector is used to indicate the state of the virtual digital human.

[0035] Figure 2: is a flow chart of a method for generating a virtual digital human provided by an embodiment of the present application. Figure 2 A method for generating a virtual digital human provided by an embodiment of the present application may include: Step 210, acquiring feature information associated with the input speech; The specific process of step 210 may refer to the above description.

[0036] Step 220, obtaining a virtual digital human service strategy based on the characteristic information; Among them, in one implementation, the operator's experience data can be obtained, and then the virtual digital human service strategy can be obtained based on the operator's experience data and the characteristic information. Of course, in the embodiment of the present application, the virtual digital human service strategy can also be directly obtained based on the characteristic information.

[0037] The virtual digital human service strategy in the embodiment of the present application may indicate the virtual digital human's service strategy for users, and the virtual digital human service strategy may include listening, comforting, guiding, praising, identifying, and sympathizing.

[0038] Step 230, based on the feature information and the virtual digital human service strategy, obtaining a virtual image driving vector set, wherein the virtual image driving vector set acts on the virtual digital human; Among them, the virtual image driving vector set can be obtained through the trained model. In this process, if the model input information is sufficient (the information covers all business types, different types of users, and different types of features, and meets the input data quality and dimensionality requirements) and the confidence of the model output (used to evaluate the accuracy and reliability of the model output) is greater than or equal to 50%, the virtual digital human service strategy associated with multiple features can be output or smoothly adjusted, and the virtual digital human service strategy and virtual image driving vector set can be creatively recommended following the changes in the user's voice and sound conditions, so as to better improve the virtual image customer service experience and improve customer satisfaction; if the model input information is insufficient or the confidence of the model output is less than 50%, it will repeat the monitoring and maintain the original strategy.

[0039] Among them, the virtual image driving vector set can be used to act on the virtual digital human by embodying the virtual digital human service strategy, for example, it can be used to drive various actions, expressions and images (including background) of the virtual digital human, and / or, the changes of various actions, expressions and images (including background). The virtual image driving vector set may include multiple vectors, each of which indicates information related to the virtual digital human. The virtual digital human service strategy may be associated with the virtual image driving vector set. For example, if the virtual digital human service strategy is listening, the vectors in the virtual image driving vector set may respectively indicate staring, chin support, blue background, etc. For example, if the virtual digital human service strategy is comforting, the vectors in the virtual image driving vector set may respectively indicate squinting, patting, warm background, etc. For example, if the virtual digital human service strategy is praise, the vectors in the virtual image driving vector set may respectively indicate smiling, liking, sensational background, etc. For example, if the virtual digital human service strategy is sympathy, the vectors in the virtual image driving vector set may respectively indicate blinking, hugging, balanced background, etc.

[0040] Step 240, determining the virtual digital human service strategy and the virtual image driving vector set as a target result associated with the virtual digital human; Step 250, obtaining a recommendation result of the image feature of the virtual digital human based on the target result and the image quantification model; Step 260, obtaining a recommendation result of the expression feature of the virtual digital human based on the target result and the expression quantification model; Step 270, obtaining a recommended result of the action feature of the virtual digital human based on the target result and the action quantification model; Step 280, based on the recommendation results of the image features of the virtual digital human, the recommendation results of the expression features of the virtual digital human, and the recommendation results of the action features of the virtual digital human, obtain the element collaborative processing results, and the element collaborative processing results are used to generate the virtual digital human.

[0041] Among them, the image quantization model, the expression quantization model and the action quantization model can all be trained deep learning models.

[0042] In the embodiment of the present application, feature information associated with the input voice is obtained; based on the feature information, a target result associated with the virtual digital human is obtained; based on the target result and the image quantification model, an image feature recommendation result of the virtual digital human is obtained; based on the target result and the expression quantification model, an expression feature recommendation result of the virtual digital human is obtained; based on the target result and the action quantification model, an action feature recommendation result of the virtual digital human is obtained; based on the image feature recommendation result of the virtual digital human, the expression feature recommendation result of the virtual digital human, and the action feature recommendation result of the virtual digital human, an element collaborative processing result is obtained, and the element collaborative processing result is used to generate the virtual digital human. In this way, in the process of generating the virtual digital human, by introducing the image quantification model, the expression quantification model and the action quantification model, the virtual digital human is quantified from multiple dimensions such as image, expression and action, and then the quantified results are collaboratively processed to obtain the element collaborative processing result, so that the generated virtual digital human integrates multiple simulation elements, which can be more natural and real compared with a single simulation element, solving the problem that the virtual digital human generated by the related technology is not natural and realistic enough.

[0043] In the embodiment of the present application, a quantization algorithm can be used in the process of generating a virtual digital human. The quantization algorithm takes various simulation elements such as different virtual digital human images, expressions, and actions as objects, classifies and quantifies their changes, and forms a proprietary feature library that can be used for the linkage control of virtual images.

[0044] The precision of the virtual image quantitative classification depends on the granularity of the sound attribute algorithm results and the precision of the virtual image simulation technology. The embodiment of the present application proposes to use the sound attribute algorithm to reversely drive the virtual image's micro-expressions, micro-movements, and micro-morphological changes, and the multi-element collaborative output can accurately convey the interlocutor's intentions and specifically appease the user's emotions. Figure 3 Discuss.

[0045] Figure 3 : is a flow chart of a method for generating a virtual digital human provided by an embodiment of the present application. Figure 1 A method for generating a virtual digital human provided by an embodiment of the present application may include: Step 310, acquiring feature information associated with the input speech; Step 320, based on the feature information, obtaining a target result associated with the virtual digital human, the target result including a virtual digital human service strategy and a virtual image driving vector set; Step 330, inputting the target result and the pre-obtained image classification into an image quantification model to obtain the image feature recommendation result of the virtual digital human; The image quantization algorithm can be used in this step. The image quantization algorithm mainly processes the image of the virtual digital person, and forms a proprietary feature library by classifying and quantifying various aspects of the image. This feature library can be used for the linkage control of the virtual image, so that the virtual image can be adjusted in real time according to specific needs. The key to the image quantization algorithm lies in the accuracy and efficiency of extracting and classifying image features.

[0046] Before step 330, the image classification may be obtained and the image quantification model may be trained. The image classification set may include body shape, gender, face shape, hair, clothing, accessories, background, etc. The image quantification model may be trained according to the output result of the voice-sound shaping algorithm, such as the target result associated with the virtual digital person, to obtain the image quantification model under different strategies. After the image quantification model is trained, the output result of the voice-sound shaping algorithm and the image classification may be input into the trained image quantification model, and a set of image feature recommendation results of the virtual digital person may be output according to the confidence, such as the image feature recommendation mark of the virtual image. If the model input information is sufficient and the confidence of the model output is greater than or equal to the threshold (e.g., 50%), the image associated with the emotional strategy may be output or smoothly adjusted to creatively realize the dynamic image interaction capability that follows the changes in the user's communication situation; if the model input information is insufficient or the confidence of the model output is less than the threshold (e.g., 50%), the original strategy may be repeatedly monitored and maintained.

[0047] Step 340, inputting the target result and the pre-obtained expression classification into an expression quantification model to obtain a recommendation result of the expression feature of the virtual digital human; In this step, the expression quantification algorithm can be used. The expression quantification algorithm focuses on the expression of the virtual digital human. By classifying and quantifying the expression, an expression feature library can be formed to drive the micro-expression changes of the virtual image. The precision of the expression quantification algorithm depends on the granularity of the sound attribute algorithm results and the precision of the virtual image simulation technology. A more sophisticated expression quantification algorithm can more accurately reflect the emotional state of the interlocutor, thereby improving the interaction effect of the virtual digital human.

[0048] Before step 340, the expression classification may be obtained and the expression quantification model may be trained. The expression classification set may include eyebrows, eyes, ears, mouth, nose, tongue, hands, feet, whole body, etc. The expression quantification model may be trained according to the output result of the sound shaping algorithm, such as the target result associated with the virtual digital human, to obtain the expression quantification model under different strategies. After the expression quantification model is trained, the output result of the sound shaping algorithm and the expression classification may be input into the trained expression quantification model, and a set of expression feature recommendation results of the virtual digital human may be output according to the confidence, such as the recommendation mark of the expression feature of the virtual image. If the model input information is sufficient and the confidence of the model output is greater than or equal to the threshold (e.g., 50%), the expression and micro-expression associated with the emotional strategy may be output or smoothly adjusted, and the dynamic expression interaction capability that changes with the user's communication situation may be creatively realized; if the model input information is insufficient or the confidence of the model output is less than the threshold (e.g., 50%), the original strategy will be repeatedly monitored and maintained.

[0049] Step 350, inputting the target result and the pre-obtained action classification into an action quantification model to obtain the action feature recommendation result of the virtual digital human; In this step, the action quantification algorithm can be used. The action quantification algorithm mainly processes the actions of the virtual digital human. By classifying and quantifying the actions, a motion feature library can be formed to drive the micro-movement changes of the virtual image. The precision of the action quantification algorithm also depends on the granularity of the sound attribute algorithm results and the precision of the virtual image simulation technology. A more sophisticated action quantification algorithm can more accurately reflect the needs and intentions of the interlocutor, thereby improving the interaction effect of the virtual digital human.

[0050] Before step 350, the action classification and action quantification model can be obtained first. Among them, the action classification set may include actions such as walking, running, jumping, standing, sitting, and customization. The action quantification model can be trained according to the output results of the voice shaping algorithm, such as the target results associated with the virtual digital human, to obtain the action quantification model under different strategies. After the action quantification model is trained, the output results of the voice shaping algorithm and the action classification can be input into the trained action quantification model, and a set of action feature recommendation results of the virtual digital human can be output according to the confidence, such as the action feature recommendation mark of the virtual image. If the model input information is sufficient and the confidence of the model output is greater than or equal to the threshold (for example, 50%), the action associated with the emotional strategy can be output or smoothly adjusted to creatively realize the action interaction capability that follows the changes in the user's communication situation; if the model input information is insufficient or the confidence of the model output is less than the threshold (for example, 50%), the original strategy will be repeatedly monitored and maintained.

[0051] Step 360, based on the recommendation results of the image features of the virtual digital human, the recommendation results of the expression features of the virtual digital human, and the recommendation results of the action features of the virtual digital human, obtain the element collaborative processing results, and the element collaborative processing results are used to generate the virtual digital human.

[0052] In the embodiment of the present application, feature information associated with the input voice is obtained; based on the feature information, a target result associated with the virtual digital human is obtained; based on the target result and the image quantification model, an image feature recommendation result of the virtual digital human is obtained; based on the target result and the expression quantification model, an expression feature recommendation result of the virtual digital human is obtained; based on the target result and the action quantification model, an action feature recommendation result of the virtual digital human is obtained; based on the image feature recommendation result of the virtual digital human, the expression feature recommendation result of the virtual digital human, and the action feature recommendation result of the virtual digital human, an element collaborative processing result is obtained, and the element collaborative processing result is used to generate the virtual digital human. In this way, in the process of generating the virtual digital human, by introducing the image quantification model, the expression quantification model and the action quantification model, the virtual digital human is quantified from multiple dimensions such as image, expression and action, and then the quantified results are collaboratively processed to obtain the element collaborative processing result, so that the generated virtual digital human integrates multiple simulation elements, which can be more natural and real compared with a single simulation element, solving the problem that the virtual digital human generated by the related technology is not natural and realistic enough.

[0053] It is important to understand that Figure 1-Figure 3 The explanations of the same or corresponding steps in the specification can be referenced to each other.

[0054] The method for generating a virtual digital human provided in an embodiment of the present application is further described below.

[0055] The embodiment of the present application provides a method for implementing the application of virtual digital human image video to intelligent interactive products, so as to enhance the simulation effects of virtual digital human image, expression, action, etc., and cooperate with voice emotion analysis, multi-element collaborative processing and barrier-free interaction and other technologies to improve the accuracy of service expression, effectively guide and appease user emotions and help to efficiently solve service problems. In order to enhance the effectiveness of communication between terminal users and operator customer service and the intelligent service experience, using virtual digital human image as a carrier and voice emotion analysis as a basis, through different proprietary features and generalized decision-making identifiers formed by voice analysis, the virtual digital human image, expression, action and other simulation elements are coordinated and linked to achieve true and accurate expression of customer service intentions, accurately convey customer service guidance, appeasement and other service countermeasures to users, and improve user satisfaction.

[0056] The method for generating a virtual digital human provided in the embodiment of the present application mainly includes three steps: implementation of a voice shaping algorithm, a virtual image multi-element quantization algorithm, and multi-element collaborative processing.

[0057] Step 1: Extract speech features, identify the emotions and intentions of speech features based on their changing trends, map the identified emotions and intentions to predefined categories, and use the trained model to obtain the virtual digital human service strategy and virtual image driving vector set.

[0058] Voice shaping refers to the process of converting the conveyed emotions and intentions into multiple elements such as the expression, action, and background of the virtual image based on the results of qualitative analysis of voice characteristics, and expressing them synchronously with the customer service voice feedback to enhance user acceptance and reduce confrontation, and better solve problems. Compared with traditional virtual digital human image technology, this proposal refines the manipulation of various elements of the virtual image, allowing the virtual image to express various postures and actions, making it more humanized and able to express more emotional intentions.

[0059] Wherein, step one may include the following sub-steps: Sub-step 1: Speech feature extraction, analyzing the basic feature change trend for adjacent time slices: The input speech is subjected to feature extraction to obtain basic speech features, such as pitch, speed, volume, etc. These features will be used to qualitatively analyze the emotions and intentions conveyed by the speech.

[0060] 1) Analyze the preprocessed target speech and select the time slice length T interval =m milliseconds, where m is a positive integer, and the basic speech feature set S1={pitch, speed, volume, ...} within a period of time is extracted for subsequent analysis of the basic change rules within different time slices.

[0061] 2) Analyze the change trend of basic features of speech for adjacent time slices. The change trend classification includes sudden increase, sudden decrease, fluctuation, etc., and extract the basic feature change trend set S2={pitch sudden increase, pitch sudden decrease, pitch fluctuation, sound speed sudden increase, sound speed sudden decrease, sound speed fluctuation, volume sudden increase, volume sudden decrease, volume fluctuation, ...} within a period of time for subsequent emotion and intention recognition. The basic feature change trend of speech mentioned in the embodiment of the present application is also called speech change trend feature.

[0062] Sub-step 2: Based on the extracted speech features, use machine learning or deep learning algorithms to identify emotions and intents: Based on the extracted speech features, machine learning or deep learning algorithms are used to identify emotions and intents. Emotions and intents are mapped to predefined categories based on speech features.

[0063] 1) Using feature sets S1 and S2, extract the first-level proprietary feature set S3 within a period of time. The first-level proprietary feature set is, for example, an emotional feature, which may include three categories: positive, neutral, and negative emotions. S3 = {positive: {politeness, tolerance, ...}, neutral: {confused, urgent, help-seeking, ...}, negative: {complaint, complaining, anxious, confrontational, ...}}.

[0064] 2) Extract the secondary proprietary feature set S4 within a period of time. The secondary proprietary feature set is, for example, a feature of emotional change, which may be deepening of negative emotions, deepening of positive emotions, maintenance of neutral emotions, etc. S4 = {deepening of negative emotions, maintenance of negative emotions, weakening of negative emotions, generation of negative emotions, abnormal fluctuation of emotions, ...}.

[0065] Sub-step 3: Based on the operator’s accumulated experience data and features S1, S2, S3, and S4, obtain the virtual digital human service strategy and virtual image driving vector set: 1) Based on the operator’s accumulated experience data and features S1, S2, S3, and S4, a set of virtual digital human service strategies ST is sorted out. ST = {listen, comfort, guide, praise, identify, sympathize, ...}.

[0066] Table 1 Example of the relationship between various feature sets and service strategies

[0067] 2) Input the features S1, S2, S3, S4 and the service strategy set ST into the trained model, and calculate and match the output vectorized virtual image driving vector set V={h1, h2, ..., h n If the model input information is sufficient or the confidence of the model output is greater than or equal to the threshold (for example, 50%), the service strategy associated with multiple features can be output or smoothly adjusted, creatively implementing the recommendation of service strategies and virtual image driving vector sets following the changes in the user's voice sound conditions, so as to better improve the image of the virtual digital person and enhance customer satisfaction; if the model input information is insufficient or the confidence of the model output is less than the threshold (for example, 50%), it will repeat monitoring and maintain the original strategy.

[0068] The mapping examples of the avatar driving vector set V are shown in Table 2. A set of service strategies ST can map multiple sets of V vectors, thereby achieving a more natural and realistic avatar customer service interaction experience.

[0069] Table 2 Example of mapping relationship between service strategy and driving vector

[0070] Step 2: Input the obtained virtual digital human service strategy and virtual image driving vector set into the trained image quantization model, expression quantization model and action quantization model respectively, and output the image feature recommendation identifier, expression feature recommendation identifier and action feature recommendation identifier of the virtual image.

[0071] The quantification algorithm takes the image, expression, movement and other simulation elements of the virtual digital human as objects, classifies and quantifies their changes, and forms a proprietary feature library that can be used for the coordinated control of the virtual digital human.

[0072] The precision of the quantitative classification of the virtual image depends on the granularity of the results of the voice attribute algorithm and the precision of the virtual image simulation technology. The embodiment of the present application creatively proposes to use the voice attribute algorithm to reversely drive the micro-expressions, micro-movements, and micro-morphological changes of the virtual image, and the multi-element collaborative output can accurately convey the interlocutor's intentions and specifically appease the user's emotions.

[0073] ① Image quantization algorithm: The image quantization algorithm mainly processes the image of the virtual digital human, and forms a proprietary feature library by classifying and quantifying various aspects of the image. This feature library can be used for the linkage control of the virtual image, so that the virtual image can be adjusted in real time according to specific needs. The key to the image quantization algorithm lies in the accuracy and efficiency of extracting and classifying image features.

[0074] 1) Obtain image multi-element classification. Let the image classification set A = {body shape, gender, face shape, hair, clothing, accessories, background, ...}.

[0075] 2) Obtain the image quantization model. According to the output results of the voice shaping algorithm, train the image quantization model under different strategies.

[0076] The output results and image classification of the voice shaping algorithm are input into the trained model, and a set of virtual image feature recommendation labels are output according to the confidence. If the input information is sufficient and the output confidence N>=50%, the image associated with the emotional strategy can be output or smoothly adjusted to creatively realize the dynamic image interaction capability that changes with the user's communication situation; if the input information is insufficient or the output confidence N<50%, the original strategy will be repeatedly monitored and maintained.

[0077] ②Expression quantization algorithm: The expression quantification algorithm focuses on the expression of the virtual digital human. By classifying and quantifying the expression, an expression feature library can be formed to drive the micro-expression changes of the virtual digital human. The precision of the expression quantification algorithm depends on the granularity of the sound attribute algorithm results and the precision of the virtual digital human simulation technology. A more sophisticated expression quantification algorithm can more accurately reflect the emotional state of the interlocutor, thereby improving the interaction effect of the virtual digital human.

[0078] 1) Obtain the multi-element classification of expression. Let the expression classification set B = {eyebrows, eyes, ears, mouth, nose, tongue, hands, feet, whole body, ...}.

[0079] 2) Obtain the expression quantization model. According to the output of the voice shaping algorithm, train the expression countermeasure model under different strategies.

[0080] The output results of the voice shaping algorithm and the expression classification are input into the trained model, and a set of virtual digital human expression feature recommendation labels are output according to the confidence level. If the input information is sufficient and the output confidence level N>=50%, the expressions and micro-expressions associated with the emotional strategy can be output or smoothly adjusted, creatively realizing the dynamic expression interaction capability that changes with the user's communication situation; if the input information is insufficient or the output confidence level N<50%, the original strategy will be repeatedly monitored and maintained.

[0081] ③Action quantization algorithm: The action quantification algorithm mainly processes the actions of the virtual digital human. By classifying and quantifying the actions, a motion feature library can be formed to drive the micro-movement changes of the virtual digital human. The precision of the action quantification algorithm also depends on the granularity of the sound attribute algorithm results and the precision of the virtual digital human simulation technology. A more sophisticated action quantification algorithm can more accurately reflect the needs and intentions of the interlocutor, thereby improving the interaction effect of the virtual digital human.

[0082] 1) Obtain action multi-element classification. Suppose the action classification set C = {walk, run, jump, stand, sit, customize, ...}.

[0083] 2) Obtain the action quantization model. According to the output of the sound shaping algorithm, train the action countermeasure model under different strategies.

[0084] The output results and action classification of the voice shaping algorithm are input into the trained model, and a set of action feature recommendation labels of virtual digital people are output according to the confidence. If the input information is sufficient and the output confidence N>=50%, the actions associated with the emotional strategy can be output or smoothly adjusted to creatively realize the action interaction capability that changes with the user's communication situation; if the input information is insufficient or the output confidence N<50%, the original strategy will be repeatedly monitored and maintained.

[0085] In summary, the quantification algorithm forms a proprietary feature library by classifying and quantifying the image, expression and action, which is used to drive the linkage control of the virtual digital human, thereby improving the interactive effect of the virtual digital human. The image quantification algorithm, expression quantification algorithm and action quantification algorithm are the three main components of the quantification algorithm, which are responsible for processing the image, expression and action of the virtual digital human respectively.

[0086] Step 3: Input the output image feature recommendation results, expression feature recommendation results, action feature recommendation results and voice shaping algorithm output results into the trained element collaboration model to output the virtual image state vector.

[0087] In this way, the asynchrony problem caused by the sound shaping algorithm driving the image, expression, action, etc. separately can be solved, and the synchronization algorithm of the above-mentioned multiple quantized simulation elements can be realized.

[0088] Sub-step 1: Construct an element coordination model. The model can be trained based on the output results of the image classification, expression classification, action classification and voice shaping algorithm to obtain a trained element coordination model. During the training process, conflicting and inharmonious element combinations can be marked to obtain a negative list. The negative list can refer to combinations of images, expressions, and actions that have conflicting state expressions. The state combinations recorded in the negative list will not appear in the virtual image state vector.

[0089] Sub-step 2: Collaborative display of multiple elements of the virtual image. Construct an algorithm function f(X, Y, Z, R, NL, Timestamp), where X is the image feature recommendation identifier, Y is the expression feature recommendation identifier, Z is the action feature recommendation identifier, R is the output result of the voice shaping algorithm, NL is the negative list, and Timestamp is the timestamp. After feature matching of the negative list, voice shaping result association verification and time verification, a set of virtual image state vectors O is output. Among them, the state vector O is the final state of the virtual image. For example, O={eyebrow: {category: willow leaf eyebrow, shape: up, degree: slight}, mouth: {category: thin, shape: up, degree: large}, action: {category: jump, shape: up, degree: medium}, ...}, after collaborative display, it reflects that the caller is in high spirits and in a good mood after solving the problem.

[0090] It should be understood that the above description can be applied to Figure 1-Figure 3 .

[0091] Compared with the prior art, the method for generating a virtual customer provided in the embodiment of the present application has the following advantages: First, voice shaping enhances barrier-free expression in all postures: Voice shaping refers to the conversion of conveyed emotions and intentions into multiple elements such as expressions, actions, and backgrounds of virtual images based on the results of qualitative analysis of voice features, and synchronously expresses them with customer service voice feedback to enhance user acceptance and reduce confrontation, so as to better solve problems. Compared with traditional virtual image technology, this proposal refines the manipulation of various elements of virtual images, allowing virtual images to express various postures and actions, making them more humanized and able to express more emotional intentions. In addition, it focuses on the synergy and change relationship between the actions, voices, lips, and backgrounds of virtual digital human images. By optimizing the interaction logic of each element, users can experience a smooth and natural interaction experience; a set of rules can be designed to describe the change frequency and trigger conditions of different elements of virtual images. For example, when a user asks a question, the animation, voice, lips, and other elements of the virtual digital image will be adjusted in real time according to pre-set rules to improve the interaction effect.

[0092] Second, the multi-element quantitative manipulation of virtual images is enhanced: the multi-element quantification algorithm takes different virtual digital human images, expressions, actions and other simulation elements as objects, classifies and quantifies their changes, and forms a proprietary feature library that can be used for the linkage manipulation of virtual images. The finer the classification and quantification granularity, the richer the manipulation expression intention. At the same time, classification and quantification will further enhance the personalized customization capabilities of virtual digital human images. It can provide users with personalized virtual digital human images based on user needs and scene characteristics.

[0093] Third, personalized customization of virtual digital human images: users can be provided with personalized virtual digital human images based on their needs and scene characteristics. For example, users can choose different virtual digital human images, animation styles, and voice characteristics according to their preferences.

[0094] Fourth, adaptive adjustment based on user behavior: The interactive logic and element changes of the virtual digital human image can be dynamically adjusted according to the user's historical behavior and preferences. For example, the user's preferences for the virtual digital human image can be analyzed, and real-time adjustments can be made based on the analysis results to improve user satisfaction.

[0095] Corresponding to the method for generating a virtual digital human provided in the embodiment of the present application, the embodiment of the present application also provides a device for generating a virtual digital human. Figure 4 is a structural block diagram of a device for generating a virtual digital human provided by an embodiment of the present application. Figure 4 The device 400 for generating a virtual digital human provided in the embodiment of the present application may include: an acquisition module 410 and a processing module 420. Among them: An acquisition module 410 is used to acquire feature information associated with the input speech; Processing module 420 is used to obtain a target result associated with the virtual digital human based on the feature information; obtain a recommendation result of the image feature of the virtual digital human based on the target result and the image quantification model; obtain a recommendation result of the expression feature of the virtual digital human based on the target result and the expression quantification model; obtain a recommendation result of the action feature of the virtual digital human based on the target result and the action quantification model; obtain an element coordination processing result based on the recommendation result of the image feature of the virtual digital human, the recommendation result of the expression feature of the virtual digital human and the recommendation result of the action feature of the virtual digital human, and the element coordination processing result is used to generate a virtual digital human.

[0096] In the device for generating a virtual digital human provided in an embodiment of the present application, in the process of generating a virtual digital human, an image quantization model, an expression quantization model and an action quantization model are introduced to quantify the virtual digital human from multiple dimensions of image, expression and action, and then the quantified results are collaboratively processed to obtain an element collaborative processing result, so that the generated virtual digital human integrates multiple simulation elements and can be more natural and realistic compared to a single simulation element, thereby solving the problem that the virtual digital human generated by the related technology is not natural and realistic enough.

[0097] In one embodiment of the present application, in the process of obtaining the target result associated with the virtual digital human based on the feature information, the processing module 420 is specifically used to: obtain the virtual digital human service strategy based on the feature information; obtain the virtual image drive vector set based on the feature information and the virtual digital human service strategy, and the virtual image drive vector set is used to act on the virtual customer by embodying the behavior of the virtual digital human service strategy; determine the virtual digital human service strategy and the virtual image drive vector set as the target result associated with the virtual digital human.

[0098] In one embodiment of the present application, in the process of obtaining a virtual image drive vector set based on the feature information and the virtual digital human service strategy, the processing module 420 is specifically used to: input the feature information and the virtual digital human service strategy into a machine learning model; obtain the virtual image drive vector set through the machine learning model; wherein the virtual image drive vector set matches the feature information and the virtual digital human service strategy.

[0099] The feature information may include at least one of basic voice features, voice change trend features, emotion features and emotion change features.

[0100] The basic speech features include at least one of pitch, speed and volume.

[0101] In one embodiment of the present application, in the process of obtaining the image feature recommendation result of the virtual digital person based on the target result and the image quantification model, the processing module 420 is specifically used to: obtain the image classification; input the target result and the image classification into the image quantification model to obtain the image feature recommendation result of the virtual digital person.

[0102] In one embodiment of the present application, in the process of obtaining the expression feature recommendation result of the virtual digital human based on the target result and the expression quantification model, the processing module 420 is specifically used to: obtain the expression classification; input the target result and the expression classification into the expression quantification model to obtain the expression feature recommendation result of the virtual digital human.

[0103] In one embodiment of the present application, in the process of obtaining the action feature recommendation result of the virtual digital human based on the target result and the action quantification model, the processing module 420 is specifically used to: obtain the action classification; input the target result and the action classification into the action quantification model to obtain the action feature recommendation result of the virtual digital human.

[0104] In one embodiment of the present application, in the process of obtaining the element collaborative processing result based on the image feature recommendation result of the virtual digital human, the expression feature recommendation result of the virtual digital human and the action feature recommendation result of the virtual digital human, the processing module 420 is specifically used to: input the image feature recommendation result of the virtual digital human, the expression feature recommendation result of the virtual digital human and the action feature recommendation result of the virtual digital human into the element collaborative model to obtain the element collaborative processing result.

[0105] The element collaboration model is trained based on image classification, expression classification, action classification and the target result.

[0106] The element collaborative processing result includes a virtual image state vector, and the virtual image state vector is used to indicate the state of the virtual digital human.

[0107] Each module in the device for generating a virtual digital human provided in the embodiment of the present application can implement each process implemented in the above method embodiment. The specific content implemented by each module can refer to the above description, and will not be repeated here to avoid repetition.

[0108] like Figure 5As shown, the embodiment of the present application also provides an electronic device 500. The electronic device 500 includes: a processor 510 and a memory 520, and the memory 520 stores a program or instruction, and the program or instruction is executed by the processor 510 to implement the steps of any of the methods described above. For example, when the program is executed by the processor 110, the following process is implemented: acquiring feature information associated with the input voice; based on the feature information, obtaining a target result associated with a virtual digital person; based on the target result and the image quantification model, obtaining an image feature recommendation result of the virtual digital person; based on the target result and the expression quantification model, obtaining an expression feature recommendation result of the virtual digital person; based on the target result and the action quantification model, obtaining an action feature recommendation result of the virtual digital person; based on the image feature recommendation result of the virtual digital person, the expression feature recommendation result of the virtual digital person, and the action feature recommendation result of the virtual digital person, obtaining an element collaborative processing result, and the element collaborative processing result is used to generate a virtual digital person. In this way, in the process of generating virtual digital humans, by introducing image quantification models, expression quantification models and action quantification models, the virtual digital humans are quantified from multiple dimensions of image, expression and action respectively, and then the quantified results are processed collaboratively to obtain the element collaborative processing results, so that the generated virtual digital humans integrate multiple simulation elements. Compared with a single simulation element, they can be more natural and realistic, solving the problem that virtual digital humans generated by related technologies are not natural and realistic enough.

[0109] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of each embodiment of the method for generating a virtual digital human are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0110] The processor is the processor in the electronic device described in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0111] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0112] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0113] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0114] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present application.

[0115] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A method for generating a virtual digital human, characterized in that: include: Acquiring feature information associated with the input speech; Based on the feature information, a target result associated with the virtual digital human is obtained; Based on the target result and the image quantification model, obtaining the image feature recommendation result of the virtual digital human; Based on the target result and the expression quantification model, obtaining the expression feature recommendation result of the virtual digital human; Based on the target result and the action quantification model, obtaining action feature recommendation results of the virtual digital human; Based on the recommendation results of the image features of the virtual digital human, the recommendation results of the expression features of the virtual digital human and the recommendation results of the action features of the virtual digital human, an element collaborative processing result is obtained, and the element collaborative processing result is used to generate a virtual digital human.

2. The method according to claim 1, characterized in that The step of obtaining a target result associated with the virtual digital human based on the feature information includes: Based on the characteristic information, obtaining a virtual digital human service strategy; Based on the feature information and the virtual digital human service strategy, a virtual image driving vector set is obtained, and the virtual image driving vector set acts on the virtual digital human; The virtual digital human service strategy and the virtual image driving vector set are determined as target results associated with the virtual digital human.

3. The method according to claim 2, characterized in that The step of obtaining a virtual image driving vector set based on the feature information and the virtual digital human service strategy includes: Inputting the feature information and the virtual digital human service strategy into a machine learning model; Obtaining a virtual image driving vector set through the machine learning model; The virtual image driving vector set matches the feature information and the virtual digital human service strategy.

4. The method according to any one of claims 1 to 3, characterized in that: The feature information includes at least one of basic voice features, voice change trend features, emotional features and emotional change features; The basic speech features include at least one of pitch, speed and volume.

5. The method according to claim 1, characterized in that The step of obtaining the image feature recommendation result of the virtual digital human based on the target result and the image quantification model includes: Get image classification; The target result and the image classification are input into an image quantification model to obtain a recommendation result of the image feature of the virtual digital human.

6. The method according to claim 1, characterized in that The step of obtaining the expression feature recommendation result of the virtual digital human based on the target result and the expression quantification model includes: Get expression classification; The target result and the expression classification are input into an expression quantification model to obtain a recommendation result of the expression feature of the virtual digital human.

7. The method according to claim 1, characterized in that The step of obtaining the action feature recommendation result of the virtual digital human based on the target result and the action quantification model includes: Get action classification; The target result and the action classification are input into an action quantification model to obtain an action feature recommendation result of the virtual digital human.

8. The method according to claim 1, characterized in that The element collaborative processing result is obtained based on the image feature recommendation result of the virtual digital person, the expression feature recommendation result of the virtual digital person and the action feature recommendation result of the virtual digital person, including: The image feature recommendation results of the virtual digital person, the expression feature recommendation results of the virtual digital person and the action feature recommendation results of the virtual digital person are input into the element coordination model to obtain the element coordination processing results.

9. The method according to claim 8, characterized in that The element collaborative processing result is a result of passing the negative list verification and the target result verification, and the negative list is a combination of elements in which at least two of the image, expression, and action are in conflict.

10. The method according to claim 8, characterized in that The element collaborative processing result includes a virtual image state vector, and the virtual image state vector is used to indicate the state of the virtual digital human.

11. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the method according to any one of claims 1 to 10 are implemented.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed, the steps of the method according to any one of claims 1 to 10 are implemented.

13. A computer program product, characterized in that The invention comprises a computer program which, when executed, implements the steps of the method according to any one of claims 1 to 10.

Images