Digital human duplicate processing method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a processing method and device for a digital human duplicate, electronic equipment and a medium, and the method comprises the steps: obtaining first video data, and determining a plurality of first image frames from the first video data; wherein the first image frame has a portrait head area and a portrait hand area; obtaining audio data, and adopting the audio data to drive a portrait head presented in a portrait head area in the first image frame to obtain a second image frame; wherein the portrait hand region in the second image frame presents a gesture the same as that of the first image frame; and according to the audio data and the second image frame, generating second video data presenting the digital human clones. According to the embodiment of the invention, the audio is adopted to drive the head action of the digital human cloned body, the digital human cloned body can present the gesture action, and the video effect of the digital human cloned body is improved.

Description

Technical Field

[0001] The present invention relates to the field of digital human technology, and in particular to a method, device, electronic equipment and medium for processing a digital human avatar. Background Art

[0002] At present, digital avatar technology is gradually being applied in some scenarios. For example, in video conferences, when users cannot attend the meeting in person, they can use digital avatars to participate in the meeting on their behalf. Digital avatars can also be used as virtual images of enterprises for product display and marketing promotion. Digital avatars can also be used as brand spokespersons for product publicity and promotion. Digital avatars have high flexibility and quantification, and can be adjusted according to enterprise needs. However, in the existing digital avatar technology, the video effect of the generated digital avatars is not good. Summary of the invention

[0003] In view of the above problems, a method, device, electronic device and medium for processing a digital human avatar are proposed to overcome the above problems or at least partially solve the above problems, including:

[0004] A method for processing a digital human avatar, the method comprising:

[0005] Acquire first video data, and determine a plurality of first image frames from the first video data; wherein the first image frames have a portrait head region and a portrait hand region;

[0006] Acquire audio data, and use the audio data to drive the head of the portrait presented in the head area of ​​the portrait in the first image frame to obtain a second image frame; wherein the hand area of ​​the portrait in the second image frame presents the same gesture as that in the first image frame;

[0007] Second video data presenting a digital human avatar is generated based on the audio data and the second image frame.

[0008] Optionally, using the audio data to drive the head of a portrait presented in the head area of ​​the portrait in the first image frame to obtain a second image frame includes:

[0009] Using the audio data, driving the head of a portrait presented in the head area of ​​the portrait in the first image frame to generate a digital head image;

[0010] The digital human head image is used to update the human head region in the first image frame to obtain a second image frame; wherein the human head region in the second image frame presents the digital human head image.

[0011] Optionally, the using the audio data to drive the portrait head presented in the portrait head area in the first image frame to generate a digital human head image includes:

[0012] generating head movement parameters according to the audio data and the first image frame;

[0013] According to the head movement parameters, the portrait head presented in the portrait head area in the first image frame is driven to generate a digital human head image.

[0014] Optionally, the head movement parameters include facial expression parameters and head posture parameters, and generating the head movement parameters according to the audio data and the first image frame includes:

[0015] Generate facial expression parameters according to the audio data and the first image frame;

[0016] Generate head posture parameters based on the first image frame.

[0017] Optionally, the facial expression parameters include mouth movement parameters and eye movement parameters, and generating the facial expression parameters according to the audio data and the first image frame includes:

[0018] generating mouth movement parameters according to the audio data and the first image frame;

[0019] Eye movement parameters are generated according to the first image frame.

[0020] Optionally, determining a plurality of first image frames from the first video data includes:

[0021] Decomposing the first video data into a plurality of first image frames according to a preset frame rate;

[0022] The step of generating second video data presenting the digital human avatar according to the audio data and the second image frame comprises:

[0023] The audio data and the second image frame are combined according to a preset frame rate to obtain second video data presenting the digital human avatar.

[0024] Optionally, the audio data is audio data sent by a target conference terminal in a video conference, and further includes:

[0025] The second video data is sent to other conference terminals.

[0026] A processing device for a digital human avatar, the device comprising:

[0027] A first image frame acquisition module, used to acquire first video data and determine a plurality of first image frames from the first video data; wherein the first image frame has a portrait head region and a portrait hand region;

[0028] a second image frame obtaining module, configured to obtain audio data, and use the audio data to drive the head of the portrait presented in the head region of the portrait in the first image frame to obtain a second image frame; wherein the hand region of the portrait in the second image frame presents the same gesture as that in the first image frame;

[0029] The second video data generating module is used to generate the second video data presenting the digital human avatar according to the audio data and the second image frame.

[0030] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method described above when executed by the processor.

[0031] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described above is implemented.

[0032] The embodiments of the present invention have the following advantages:

[0033] In an embodiment of the present invention, by acquiring first video data and determining multiple first image frames from the first video data, the first image frame having a portrait head area and a portrait hand area, acquiring audio data, and using the audio data to drive the portrait head presented in the portrait head area in the first image frame, a second image frame is obtained, and the portrait hand area in the second image frame presents the same gesture as the first image frame. According to the audio data and the second image frame, second video data presenting a digital human avatar is generated, thereby realizing the use of audio to drive the head movement of the digital human avatar and enabling the digital human avatar to present gesture movements, thereby improving the video effect of the digital human avatar. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0035] Figure 1 is a flowchart of the steps of a method for processing a digital human avatar provided by some embodiments of the present invention;

[0036] Figure 2is a schematic diagram of a system architecture provided by some embodiments of the present invention;

[0037] Figure 3 is a flowchart of the steps of another method for processing a digital human avatar provided by some embodiments of the present invention;

[0038] Figure 4 is a flowchart of the steps of another method for processing a digital human avatar provided by some embodiments of the present invention;

[0039] Figure 5 is a flowchart of the steps of another method for processing a digital human avatar provided by some embodiments of the present invention;

[0040] Figure 6 It is a structural block diagram of a digital human avatar processing device provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0041] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Reference Figure 1 , shows a flowchart of a method for processing a digital human avatar provided by some embodiments of the present invention, which may specifically include the following steps:

[0043] Step 101, obtaining first video data, and determining a plurality of first image frames from the first video data; wherein the first image frames have a portrait head region and a portrait hand region.

[0044] In practical applications, the first video data may be pre-recorded and stored, and the first video data may be used as the initial video for subsequent generation of the digital human avatar video.

[0045] After obtaining the first video data, multiple first image frames contained in the first video data can be extracted. The first image frame can have a portrait head area and a portrait hand area. Of course, the first image frame also has other portrait areas, such as a portrait body area, a portrait leg area, etc.

[0046] In some examples, the first video data may be a full-body video with gestures, no lip shape, and no facial expressions, such as a video recorded by a user with gestures and no changes in lip shape.

[0047] Since the first video data is a video with gestures but no lip movements and no facial expressions, the hand area of ​​the portrait in the first image frame shows gestures, while the head area of ​​the portrait in the first image frame is a relatively static picture, that is, the head of the portrait is shown but without any movement, such as the mouth and eyes have no movement.

[0048] In some embodiments of the present invention, determining a plurality of first image frames from the first video data includes: decomposing the first video data into a plurality of first image frames according to a preset frame rate.

[0049] As an example, the preset frame rate is 25 FPS (Frame Per Second).

[0050] In practical applications, the first video data may be decomposed into multiple first image frames according to a preset frame rate, and may be saved in a specified directory, and the decomposed multiple first image frames may be numbered, such as 1, 2, 3, ..., 999, 1000. In some examples, the multiple first image frames may be numbered and stored according to the time sequence of the multiple first image frames in the first video data.

[0051] Step 102, obtain audio data, and use the audio data to drive the portrait head presented in the portrait head area in the first image frame to obtain a second image frame; wherein the portrait hand area in the second image frame presents the same gesture as that in the first image frame.

[0052] As an example, the audio data may be audio data sent by a target conference terminal in a video conference, or may be recorded audio data, such as recorded audio for promotion, explanation, or product introduction.

[0053] Since the head area of ​​the portrait in the first image frame is a relatively static picture, showing the head of the portrait, but without any movement, such as the mouth and eyes, the audio data can be used to drive the head of the portrait presented in the head area of ​​the portrait in the first image frame, such as driving the mouth of the portrait head to change synchronously with the audio. Moreover, since the hand area of ​​the portrait in the first image frame shows a gesture, the original gesture can be combined to obtain a second image frame with a driven head of the portrait and a gesture.

[0054] In some examples, since the first video data is decomposed into multiple first image frames according to a preset frame rate, the preset frame rate can represent the number of image frames corresponding to the audio clip per second. Therefore, a series of image frames can be driven according to the correspondence between each first image frame and the audio clip. For example, if the preset frame rate is 25FPS, 25 first image frames can be driven for every 1-second audio clip.

[0055] In some embodiments of the present invention, using the audio data to drive the head of a portrait presented in the head area of ​​the portrait in the first image frame to obtain the second image frame includes:

[0056] The audio data is used to drive the portrait head presented in the portrait head area in the first image frame to generate a digital human head image; the digital human head image is used to update the portrait head area in the first image frame to obtain a second image frame; wherein the portrait head area in the second image frame presents a digital human head image.

[0057] In actual applications, a digital human head image can be generated by driving the portrait head presented in the portrait head area in the first image frame. The digital human head image can then be used to update the portrait head area in the first image frame, thereby making the digital human head image fit with other portrait areas of the first image frame to form a second image frame presenting the digital human head image and gestures.

[0058] In some embodiments of the present invention, using the audio data to drive the portrait head presented in the portrait head area in the first image frame to generate a digital human head image includes:

[0059] Generate head motion parameters according to the audio data and the first image frame; and drive the portrait head presented in the portrait head area in the first image frame according to the head motion parameters to generate a digital human head image.

[0060] In some examples, audio features may be extracted from audio data, and the audio features may be used to drive the portrait head presented in the portrait head region in the first image frame.

[0061] In actual applications, based on the audio features and the first image frame, the expression module and the action module generate head motion parameters corresponding to the audio features, such as expression vectors and action vectors. The head motion parameters are then input into the rendering module, and the head motion parameters are used for rendering based on the first image frame to generate a digital human head image that shows the lip shape and expression movements corresponding to the audio.

[0062] As an example, the head motion parameter may be a 3DMM (3D Morphable Model, a three-dimensional face statistical model) coefficient, that is, a 3DMM intermediate feature.

[0063] 3DMM coefficients are a set of parameters used to describe and generate the shape and texture of a 3D face. By adjusting these coefficients, a 3D face model with different shapes, textures, and expressions can be created. These coefficients are widely used in face recognition, face reconstruction, animation, and other fields. For example, shape coefficients (α and β): used to adjust the shape of the face, where α represents the coefficient of the identity characteristics of the person, and β represents the coefficient of the expression. These coefficients are multiplied and added with the shape basis vector to obtain a specific face shape; texture coefficient (δ): used to adjust the texture of the face, that is, the color and details on the skin. After multiplying and adding with the texture basis vector, a specific face texture can be generated.

[0064] In some embodiments of the present invention, the head motion parameters include facial expression parameters and head posture parameters, and generating the head motion parameters according to the audio data and the first image frame includes:

[0065] Generate facial expression parameters based on the audio data and the first image frame; and generate head posture parameters based on the first image frame.

[0066] In some embodiments of the present invention, the facial expression parameters include mouth movement parameters and eye movement parameters, and generating the facial expression parameters according to the audio data and the first image frame includes:

[0067] Generate mouth movement parameters based on the audio data and the first image frame; and generate eye movement parameters based on the first image frame.

[0068] In practical applications, the head motion parameters used to drive the portrait head may include facial expression parameters and head posture parameters. The facial expression parameters may include mouth motion parameters and eye motion parameters, such as Figure 2 ,The head posture parameters are used to control the head posture, the mouth movement parameters are used to control the mouth movement, and the eye movement parameters are used to control the eye movement.

[0069] The head posture parameters can be directly generated based on the first image frame, that is, the head posture in the first image frame is reused. The eye movement parameters can also be directly generated based on the first image frame, that is, the eye movement in the first image frame is reused. The mouth movement parameters can be generated based on the audio data and the first image frame, that is, based on the first image frame, the audio data is used to drive the mouth movement.

[0070] Among the related technologies, some digital human technologies mainly focus on facial expressions and movements, and the effects of gestures and 3D head rotation are slightly poor, resulting in unsmooth gestures and head rotation, or even no gestures and head rotation. Even if the portrait in the original image has two limbs, hand movements cannot be achieved, appearing stiff and monotonous.

[0071] In the embodiment of the present invention, by pre-recording the first video data and utilizing the digital avatar technology, the first video data is driven by specific audio, so that the mouth shape changes synchronously with the audio, and can be matched with the original gestures and head posture of the first video data, so that the digital human avatar video no longer has only facial movements without any gestures, and a more vivid and realistic digital human avatar video with gestures, facial expressions, and head movements is achieved, thereby improving the effect of the digital human avatar video.

[0072] Step 103: Generate second video data presenting the digital human avatar based on the audio data and the second image frame.

[0073] In some examples, the second video data can be used in a video conference to be sent to other conference terminals, can also be used as a virtual image of the enterprise for product display and marketing promotion, and can also be used as a brand spokesperson for product publicity and promotion.

[0074] After the second image frame is generated, the audio data and the second image frame may be combined, such as combining them in chronological order, to generate second video data, namely, the digital human avatar video.

[0075] Among them, the digital human avatar is a virtual character that combines artificial intelligence, virtual reality and other technologies. It represents an individual or entity in real life. It can be customized according to the user's personal information and preferences, and has similar appearance, voice and behavioral characteristics to the user. It relies on advanced artificial intelligence algorithms and a large amount of user data, and uses deep neural networks for image synthesis to achieve highly realistic virtual human effects, which can provide users with a more convenient and rich digital life experience.

[0076] In some embodiments of the present invention, generating second video data presenting a digital human avatar according to the audio data and the second image frame includes:

[0077] The audio data and the second image frame are combined according to a preset frame rate to obtain second video data presenting the digital human avatar.

[0078] In practical applications, since the first video data is decomposed into multiple first image frames according to a preset frame rate, the preset frame rate can represent the number of image frames corresponding to the audio clip per second, such as a preset frame rate of 25 FPS, that is, each 1-second audio clip can drive 25 first image frames. The second image frames are updated based on the first image frames, and the number of the second image frames is the same as that of the first image frames.

[0079] When performing video synthesis, the second image frame and the audio data may be combined into second video data, namely, the digital human avatar video, at a preset frame rate.

[0080] In some embodiments of the present invention, the audio data is audio data sent by a target conference terminal in a video conference, and the method further includes: sending the second video data to other conference terminals.

[0081] In a video conferencing scenario, the audio data sent by the target conference terminal in the video conference can be collected, and then the image frame in the first video data can be driven according to the audio data to obtain the second video data, and then the second video data can be sent to other conference terminals. When the user is unable to attend the meeting in person, he can participate in the meeting through a digital human avatar on his behalf.

[0082] In an embodiment of the present invention, by acquiring first video data and determining multiple first image frames from the first video data, the first image frame having a portrait head area and a portrait hand area, acquiring audio data, and using the audio data to drive the portrait head presented in the portrait head area in the first image frame, a second image frame is obtained, and the portrait hand area in the second image frame presents the same gesture as the first image frame. According to the audio data and the second image frame, second video data presenting a digital human avatar is generated, thereby realizing the use of audio to drive the head movement of the digital human avatar and enabling the digital human avatar to present gesture movements, thereby improving the video effect of the digital human avatar.

[0083] Reference Figure 3 , shows a flowchart of another method for processing a digital human avatar provided by some embodiments of the present invention, which may specifically include the following steps:

[0084] Step 301, obtaining first video data, and determining a plurality of first image frames from the first video data; wherein the first image frames have a portrait head region and a portrait hand region.

[0085] Step 302: Acquire audio data, and generate head motion parameters according to the audio data and the first image frame.

[0086] Step 303: driving the portrait head presented in the portrait head area in the first image frame according to the head movement parameters to generate a digital human head image.

[0087] Step 304, using the digital human head image, updating the portrait head area in the first image frame to obtain a second image frame; wherein the portrait head area in the second image frame presents the digital human head image, and the portrait hand area in the second image frame presents the same gesture as the first image frame.

[0088] Step 305: Generate second video data presenting the digital human avatar based on the audio data and the second image frame.

[0089] Reference Figure 4 , shows a flowchart of another method for processing a digital human avatar provided by some embodiments of the present invention, which may specifically include the following steps:

[0090] Step 401, obtaining first video data, and decomposing the first video data into a plurality of first image frames according to a preset frame rate; wherein the first image frame has a portrait head region and a portrait hand region.

[0091] Step 402, obtain audio data, and use the audio data to drive the portrait head presented in the portrait head area in the first image frame to obtain a second image frame; wherein the portrait hand area in the second image frame presents the same gesture as that in the first image frame.

[0092] Step 403: merge the audio data and the second image frame according to a preset frame rate to obtain second video data presenting the digital human avatar.

[0093] Reference Figure 5 , shows a flowchart of another method for processing a digital human avatar provided by some embodiments of the present invention, which may specifically include the following steps:

[0094] Step 501, obtaining first video data, and determining a plurality of first image frames from the first video data; wherein the first image frames have a portrait head region and a portrait hand region.

[0095] Step 502, obtain audio data sent by the target conference terminal in the video conference, and use the audio data to drive the portrait head presented in the portrait head area in the first image frame to obtain a second image frame; wherein the portrait hand area in the second image frame presents the same gesture as the first image frame.

[0096] Step 503: Generate second video data presenting the digital human avatar based on the audio data and the second image frame.

[0097] Step 504: Send the second video data to other conference terminals.

[0098] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0099] Reference Figure 6, shows a schematic diagram of the structure of a processing device for a digital human avatar provided by some embodiments of the present invention, which may specifically include the following modules:

[0100] A first image frame acquisition module 601 is used to acquire first video data and determine a plurality of first image frames from the first video data; wherein the first image frame has a portrait head region and a portrait hand region;

[0101] The second image frame obtaining module 602 is used to obtain audio data, and use the audio data to drive the head of the portrait presented in the head area of ​​the portrait in the first image frame to obtain a second image frame; wherein the hand area of ​​the portrait in the second image frame presents the same gesture as that in the first image frame;

[0102] The second video data generating module 603 is used to generate second video data presenting the digital human avatar according to the audio data and the second image frame.

[0103] In some embodiments of the present invention, using the audio data to drive the head of a portrait presented in the head area of ​​the portrait in the first image frame to obtain the second image frame includes:

[0104] Using the audio data, driving the head of a portrait presented in the head area of ​​the portrait in the first image frame to generate a digital head image;

[0105] The digital human head image is used to update the human head region in the first image frame to obtain a second image frame; wherein the human head region in the second image frame presents the digital human head image.

[0106] In some embodiments of the present invention, using the audio data to drive the portrait head presented in the portrait head area in the first image frame to generate a digital human head image includes:

[0107] generating head movement parameters according to the audio data and the first image frame;

[0108] According to the head movement parameters, the portrait head presented in the portrait head area in the first image frame is driven to generate a digital human head image.

[0109] In some embodiments of the present invention, the head motion parameters include facial expression parameters and head posture parameters, and generating the head motion parameters according to the audio data and the first image frame includes:

[0110] Generate facial expression parameters according to the audio data and the first image frame;

[0111] Generate head posture parameters based on the first image frame.

[0112] In some embodiments of the present invention, the facial expression parameters include mouth movement parameters and eye movement parameters, and generating the facial expression parameters according to the audio data and the first image frame includes:

[0113] generating mouth movement parameters according to the audio data and the first image frame;

[0114] Eye movement parameters are generated according to the first image frame.

[0115] In some embodiments of the present invention, determining a plurality of first image frames from the first video data includes:

[0116] Decomposing the first video data into a plurality of first image frames according to a preset frame rate;

[0117] The step of generating second video data presenting the digital human avatar according to the audio data and the second image frame comprises:

[0118] The audio data and the second image frame are combined according to a preset frame rate to obtain second video data presenting the digital human avatar.

[0119] In some embodiments of the present invention, the audio data is audio data sent by a target conference terminal in a video conference, and further includes:

[0120] The second video data sending module is used to send the second video data to other conference terminals.

[0121] In an embodiment of the present invention, by acquiring first video data and determining multiple first image frames from the first video data, the first image frame having a portrait head area and a portrait hand area, acquiring audio data, and using the audio data to drive the portrait head presented in the portrait head area in the first image frame, a second image frame is obtained, and the portrait hand area in the second image frame presents the same gesture as the first image frame. According to the audio data and the second image frame, second video data presenting a digital human avatar is generated, thereby realizing the use of audio to drive the head movement of the digital human avatar and enabling the digital human avatar to present gesture movements, thereby improving the video effect of the digital human avatar.

[0122] Some embodiments of the present invention further provide an electronic device, comprising a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the above method is implemented when the computer program is executed by the processor.

[0123] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0124] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above method when executed by a processor.

[0125] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0127] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0128] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0129] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0130] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0131] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0132] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0133] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0134] The above is a detailed introduction to the processing method, device, electronic device and medium of a digital human avatar. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A method for processing a digital human avatar, characterized in that: The method comprises: Acquire first video data, and determine a plurality of first image frames from the first video data; wherein the first image frames have a portrait head region and a portrait hand region; Acquire audio data, and use the audio data to drive the head of the portrait presented in the head area of ​​the portrait in the first image frame to obtain a second image frame; wherein the hand area of ​​the portrait in the second image frame presents the same gesture as that in the first image frame; Second video data presenting a digital human avatar is generated based on the audio data and the second image frame.

2. The method according to claim 1, characterized in that: Using the audio data to drive the head of a portrait presented in the head area of ​​the portrait in the first image frame to obtain a second image frame, including: Using the audio data, driving the head of a portrait presented in the head area of ​​the portrait in the first image frame to generate a digital head image; The digital human head image is used to update the human head region in the first image frame to obtain a second image frame; wherein the human head region in the second image frame presents the digital human head image.

3. The method according to claim 2, characterized in that The step of using the audio data to drive the head of a person in the head region of the first image frame to generate a digital head image of the person includes: generating head movement parameters according to the audio data and the first image frame; According to the head movement parameters, the portrait head presented in the portrait head area in the first image frame is driven to generate a digital human head image.

4. The method according to claim 3, characterized in that The head movement parameters include facial expression parameters and head posture parameters, and the step of generating the head movement parameters according to the audio data and the first image frame includes: Generate facial expression parameters according to the audio data and the first image frame; Generate head posture parameters based on the first image frame.

5. The method according to claim 4, characterized in that The facial expression parameters include mouth movement parameters and eye movement parameters, and the facial expression parameters are generated according to the audio data and the first image frame, including: generating mouth movement parameters according to the audio data and the first image frame; Eye movement parameters are generated according to the first image frame.

6. The method according to any one of claims 1 to 5, characterized in that: The determining a plurality of first image frames from the first video data comprises: Decomposing the first video data into a plurality of first image frames according to a preset frame rate; The step of generating second video data presenting the digital human avatar according to the audio data and the second image frame comprises: The audio data and the second image frame are combined according to a preset frame rate to obtain second video data presenting the digital human avatar.

7. The method according to claim 1, characterized in that The audio data is the audio data sent by the target conference terminal in the video conference, and also includes: The second video data is sent to other conference terminals.

8. A processing device for a digital human avatar, characterized in that: The device comprises: A first image frame acquisition module, used to acquire first video data and determine a plurality of first image frames from the first video data; wherein the first image frame has a portrait head region and a portrait hand region; a second image frame obtaining module, configured to obtain audio data, and use the audio data to drive the head of the portrait presented in the head region of the portrait in the first image frame to obtain a second image frame; wherein the hand region of the portrait in the second image frame presents the same gesture as that in the first image frame; The second video data generating module is used to generate the second video data showing the digital human avatar according to the audio data and the second image frame.

9. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the method according to any one of claims 1 to 7 when executed by the processor.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images