Generation method and device of 2d digital human action library, cluster and storage medium

By obtaining real-person image files and reference action libraries, a basic action library is generated, and a 2D real-person action generation model and proprietary action annotation interface is used to automatically build the action library, which solves the problems of low generation efficiency and unstandard actions of 2D digital human action library. Achieve efficient and natural action library generation.

CN119091015APending Publication Date: 2024-12-06HUAWEI CLOUD COMPUTING TECHNOLOGIES CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202311100254.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-06-05
Filing Date
2023-08-29
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

The existing 2D digital human action library is inefficient in generating, and requires manual re-recording of the actions, resulting in low efficiency and unstandard movements.

Method used

By obtaining real-person image files and reference action libraries, a basic action library is generated, and a 2D real-person action generation model and proprietary action annotation interface is used to automatically build the action library to avoid repeated recordings and ensure the consistency and nature of the action.

Benefits of technology

The generation efficiency of the 2D digital human action library is improved, ensuring the consistency and naturalness of the action library, reducing costs, and solving the problem of unstandard movements.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091015A_ABST
    Figure CN119091015A_ABST
Patent Text Reader

Abstract

The invention provides a 2d digital human action library generation method and device, a cluster and a storage medium, and belongs to the technical field of 2d digital human. The method comprises the steps that a first image file and a reference action library of a first image in a target scene are acquired, the first image file comprises a 2d real person photo of a second image or a 2d real person action video of the second image, the reference action library comprises a plurality of 2d digital human actions of the first image, and the second image is acquired based on the first image file and the reference action library. A basic action library of the second image in the target scene is generated, the basic action library comprises a plurality of 2d digital human actions, the 2d digital human actions in the basic action library are the same as the 2d digital human actions in the reference action library, and the basic action library is used for generating a 2d digital human action video of the second image. By adopting the method disclosed by the invention, the action library can be automatically obtained when the 2d digital human action library is obtained, and the obtaining efficiency of the 2d digital human action library is improved.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] This disclosure claims priority to Chinese patent application No. 202310658030.0, filed on June 5, 2023, entitled “Method, device, cluster and storage medium for generating image action library”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0002] The present disclosure relates to the technical field of 2D digital human, and in particular to a method, device, cluster and storage medium for generating a 2D digital human action library. Background Art

[0003] With the development of two-dimensional (2D) digital human technology, 2D digital human action videos are used in more and more scenes, such as speech scenes, news broadcast scenes, and short video scenes. Before generating 2D digital human action videos, an action library is pre-built, which includes multiple 2D digital human actions, such as waving, shaking hands, and making fists. When generating 2D digital human action videos, the actions in the action library are applied to the virtual character to obtain 2D digital human action videos.

[0004] In the related art, when constructing an action library, the corresponding action videos are manually recorded to obtain the action library. Thus, each time an image is constructed, the existing actions need to be re-recorded to obtain the action library, resulting in a relatively low efficiency in obtaining the action library. Summary of the invention

[0005] The present disclosure provides a method, device, cluster and storage medium for generating a 2D digital human action library, which can improve the efficiency of obtaining the action library.

[0006] In a first aspect, the present disclosure provides a method, device, cluster and storage medium for generating a 2D digital human action library, the method comprising: obtaining a first image file and a reference action library of a first image in a target scene, wherein the first image file comprises a 2D real-life photo of a second image or a 2D real-life action video of the second image, and the reference action library comprises multiple 2D digital human actions of the first image; based on the first image file and the reference action library, generating a basic action library of the second image in the target scene, wherein the basic action library comprises multiple 2D digital human actions of the second image, the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library, and the basic action library is used to generate a 2D digital human action video of the second image.

[0007] In the solution disclosed in the present disclosure, given a real-life picture or action video, a 2D digital human action library that is consistent with the action in the reference action library can be generated, without the need to re-record the 2D digital human action corresponding to the real person, thereby improving the efficiency of action library generation. Moreover, for different images, the same action can meet the standard and remain consistent.

[0008] In an optional manner, generating a basic action library of the second image in the target scene based on the first image file and the reference action library includes: inputting the 2D digital human actions in the first image file and the reference action library into a 2D real-life action generation model to generate multiple 2D digital human actions of the second image, wherein the multiple 2D digital human actions of the second image constitute the basic action library, and the 2D real-life action generation model is obtained based on training of multiple action videos.

[0009] In an optional manner, the first image file includes a real-life action video of the second image; before inputting the first image file and the 2D digital human actions in the reference action library into the 2D real-life action generation model, the method also includes: using the 2D real-life action video to update the 2D real-life action generation model.

[0010] In the solution shown in the present disclosure, before using the 2D real-person action generation model, it is updated using a 2D real-person action video, which can make the 2D digital human movements generated by the 2D real-person action generation model closer to real people.

[0011] In an optional manner, the first image file includes a 2D real-life action video of the second image; the method further includes: obtaining proprietary action information corresponding to the 2D real-life action video, wherein the proprietary action information includes a time period of the proprietary action in the 2D real-life action video; based on the 2D real-life action video and the proprietary action information, generating a proprietary action library of the second image in the target scene, wherein the proprietary action library includes the proprietary actions of the 2D digital human of the second image. In this way, the proprietary actions of the 2D digital human of the second image can also be generated, so that the actions included in the action library of the second image are more comprehensive.

[0012] In an optional manner, the obtaining of the proprietary action information corresponding to the 2D real-life action video includes: providing a proprietary action annotation interface, wherein the proprietary action annotation interface is used by the user to annotate the proprietary action of the 2D real-life action video; and obtaining the proprietary action information fed back by the user through the proprietary action annotation interface. In this way, providing the proprietary action annotation interface allows the user to annotate the proprietary action, making the proprietary action more accurate.

[0013] In an optional manner, obtaining the reference action library includes: receiving the reference action library input by a user; or receiving a scene identifier of the target scene input by a user, and determining the reference action library corresponding to the scene identifier.

[0014] In an optional manner, after generating a basic action library of the second image in the target scene, the method further includes: obtaining a driving voice and a second image file, wherein the second image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image; based on the driving voice and the second image file, generating a 2D digital human action video matching the driving voice; obtaining an action segment to be edited of the 2D digital human action video; and using the basic action library, editing the action segment to be edited in the 2D digital human action video to obtain the edited 2D digital human action video.

[0015] In the solution disclosed in the present invention, after obtaining the basic action library of the second image in the target scene, a 2D digital human action video of the second image in the target scene can be automatically generated, thereby improving the generation efficiency of the 2D digital human action video.

[0016] In a second aspect, the present disclosure provides a device for generating a 2D digital human action library, the device comprising at least one module, and the at least one module is used to implement the method for generating a 2D digital human action library provided by the above-mentioned first aspect or any optional method in the first aspect.

[0017] In some embodiments, the modules in the 2D digital human action library generation device are implemented by software, and the modules in the 2D digital human action library generation device are program modules. In other embodiments, the modules in the 2D digital human action library generation device are implemented by hardware or firmware.

[0018] In a third aspect, the present disclosure provides a computing device cluster, which includes at least one computing device, each computing device including a processor and a memory, and the processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method for generating a 2D digital human action library provided in the above-mentioned first aspect or any optional method in the first aspect.

[0019] In a fourth aspect, the present disclosure provides a computer-readable storage medium, which includes computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster executes the method for generating a 2D digital human action library provided by the above-mentioned first aspect or any optional method in the first aspect.

[0020] In a fifth aspect, the present disclosure provides a computer program product comprising instructions, which, when executed by a computing device cluster, enables the computing device cluster to execute the method for generating a 2D digital human action library provided in the first aspect or any optional method in the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 is a schematic diagram of a system architecture provided by an exemplary embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of a system architecture provided by an exemplary embodiment of the present disclosure;

[0023] Figure 3 is a flow chart of a method for generating a 2D digital human action library provided by an exemplary embodiment of the present disclosure;

[0024] Figure 4 is a flow chart of a method for generating a 2D digital human action library provided by an exemplary embodiment of the present disclosure;

[0025] Figure 5 is a flow chart of a method for generating a 2D digital human action library provided by an exemplary embodiment of the present disclosure;

[0026] Figure 6 It is a structural schematic diagram of a device for generating a 2D digital human action video provided by an exemplary embodiment of the present disclosure;

[0027] Figure 7 is a schematic diagram of the structure of a computing device provided by an exemplary embodiment of the present disclosure;

[0028] Figure 8 is a schematic diagram of the structure of a computing device cluster provided by an exemplary embodiment of the present disclosure;

[0029] Fig. 9 It is a connection diagram of a computing device cluster provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] In order to make the objectives, technical solutions and advantages of the present disclosure more clear, the embodiments of the present disclosure will be further described in detail below with reference to the accompanying drawings.

[0031] Some terminology concepts involved in the embodiments of the present disclosure are explained below.

[0032] 1. 2D digital human: is a 2D image generated by computer. 2D digital human has human appearance, with specific features such as appearance, gender and personality, and has human behavior, the ability to express with language, facial expressions and body movements, and has human thoughts, the ability to recognize the external environment, and the ability to communicate and interact with people. 2D digital human represents a virtual character, also known as 2D virtual digital human. For 2D digital human, from the perspective of presentation, users can only browse 2D digital human from a single perspective. The perspective has been determined when it was made, and users cannot change the perspective by themselves.

[0033] 2. Deep learning algorithm: It is a branch of machine learning. It is an algorithm that uses artificial neural networks as the architecture to represent and learn data.

[0034] 3. 2D digital human action library: It is a database including multiple 2D digital human actions. 2D digital human actions are action videos. These 2D digital human actions can be called and applied to 2D digital humans when needed to obtain 2D digital human action videos. 2D digital humans can also be called 2D virtual humans or 2D hyper-realistic digital humans.

[0035] 4. Digital Human Motion Editing: Modify and edit the digital human motion, including but not limited to motion splicing, adjustment and editing, to achieve specific motion performance effects. Digital Human Motion Editing can make the digital human motion more natural and smooth, meeting specific needs.

[0036] The following describes the background, execution subject, system architecture, method flow and device of the embodiments of the present disclosure in sequence. First, the background is described.

[0037] In 2D digital human technology, when building an action library, it is necessary to manually record the action video of the corresponding action, and every time a new 2D digital human is built, the real person of the new 2D digital human needs to re-record the existing action. Moreover, the results of recordings by different real people are different. In this way, each time a new 2D digital human is added, it needs to be re-recorded, which is difficult to implement on a large scale. In addition, the same action of different 2D digital humans is obviously different, and the recorded action data may not be accurate enough, and there is a problem of non-standard actions.

[0038] However, in the disclosed embodiment, given a real person's image file, a 2D digital human action library consistent with the action in the reference action library can be generated without re-recording the 2D digital human action library corresponding to the real person. Moreover, for different images, the same action can meet the standard and remain consistent.

[0039] The following describes the execution subject of the method for generating a 2D digital human action library.

[0040] The execution subject of the method is a generating device of the 2D digital human action library, referred to as generating device. Optionally, the generating device is a hardware device, which is a computing device such as a server or terminal device. Optionally, the generating device is a software device, such as a set of software programs running on the computing device.

[0041] The system architecture of the embodiment of the present disclosure is described below.

[0042] In an optional manner, the present disclosure embodiment provides a system architecture 100. Figure 1 As shown, the system architecture 100 includes a terminal device 101 and a computing device 102. The terminal device 101 and the computing device 102 are connected via a wired or wireless network. The terminal device 101 is a device used by a user, such as a desktop computer, a laptop computer, a tablet computer or a mobile phone, etc. The terminal device 101 is used for the user to interact with the computing device 102. For example, the user can input scene information and image files, or input reference action libraries and image files, etc. through the terminal device 101. The computing device 102 can be a server, etc., and the computing device 102 is used to generate a 2D digital human action library using the content input by the user. The 2D digital human action library is returned to the terminal device 101.

[0043] In another optional manner, the embodiment of the present disclosure provides another system architecture 200. Figure 2 As shown, the system architecture 200 includes a terminal device 101 and a public cloud 201. The terminal device 101 is connected to the public cloud 201 via a wired or wireless network. The terminal device 101 is a device used by the user, and the terminal device 101 is used for the user to interact with the public cloud 201. The public cloud 201 is an entity that uses basic resources to provide cloud services to users in a cloud computing model. The public cloud 201 can also be considered as a cloud environment. The public cloud 201 includes a cloud data center, which includes a large number of basic resources owned by a cloud service provider. The large number of basic resources include computing resources, storage resources, and network resources. The computing resources included in the cloud data center can be a computing device cluster, and the computing device cluster includes at least one computing device 102. The computing device 102 can be a server, etc. When a user uses a cloud service, the user can input scene information and image files through an application programming interface (API) or an interactive interface (the interactive interface can be a graphical user interface (GUI)), or input a reference action library and image files to the public cloud 201, etc. The computing device cluster in the public cloud 201 receives the content input by the user, and uses the content input by the user to generate a 2D digital human action library. The 2D digital human action library is returned to the terminal device 101.

[0044] In the system architecture 200, the method for generating the 2D digital human action library can be abstracted into a cloud service by the cloud service provider in the public cloud 201 and provided to users. After the user obtains the permission to use the cloud service, the cloud service can be used to generate the action library. The cloud service is an action library generation service.

[0045] In another optional implementation, the embodiment of the present disclosure further provides a system architecture. The system architecture includes a terminal device, which is a device used by a user. The terminal device executes the method for generating a 2D digital human action library in the embodiment of the present disclosure.

[0046] The following describes the process of the method for generating the 2D digital human action library in the embodiment of the present disclosure.

[0047] Figure 3 Provides a 2D digital human action library generation method flow, see Figure 1 Step 301 to step 303, Figure 3 In the previous text Figure 2 The system architecture shown in the figure is used as an example. Figure 3 In the process shown, the terminal device is the terminal device 101 mentioned above, and the public cloud is the public cloud 201 mentioned above.

[0048] Step 301, obtaining a first image file and a reference action library of the first image in a target scene, wherein the first image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image, and the reference action library includes multiple 2D digital human actions of the first image, and the starting postures and ending postures of the multiple 2D digital human actions are the same.

[0049] In this embodiment, the target scene is any scene, such as a speech scene and a short video live broadcast scene. In the target scene, the first image corresponds to a reference action library, which includes common 2D digital human actions in the target scene. Each 2D digital human action is an action video. The first image is a 2D digital human image of real person A. In the target scene, the second image does not correspond to an action library. The second image is a 2D digital human image of real person B. The second image is different from the first image.

[0050] The terminal device is provided with an input interface, through which the user inputs the reference action library and the first image file, the first image file including a 2D real-life photo of the second image or a 2D real-life action video of the second image. The 2D real-life photo is a full-body photo or a half-body photo, which matches the target scene. For example, when the target scene is a standing speech scene, the 2D real-life photo is a full-body photo of the second image; when the target scene is a sitting speech scene, the 2D real-life photo is a half-body photo of the second image, and the 2D real-life action video is a video of the second image with real people in action, which is recorded. The terminal device sends the reference action library and the first image file to the public cloud. The public cloud receives the reference action library and the first image file.

[0051] Alternatively, the terminal device is provided with an input interface, through which the user inputs the first image file and the scene identifier of the target scene, and the terminal device sends the first image file and the scene identifier of the target scene to the public cloud. The public cloud receives the first image file and the scene identifier. The public cloud stores the correspondence between the scene identifier and the reference action library, and the public cloud determines the reference action library corresponding to the scene identifier of the target scene in the correspondence.

[0052] Step 302, based on the first image file and the reference action library, generate a basic action library of the second image in the target scene, wherein the basic action library includes multiple 2D digital human actions of the second image, the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library, and the basic action library is used to generate a 2D digital human action video of the second image.

[0053] In this embodiment, the public cloud migrates the 2D digital human actions of the reference action library to the first image file, generates multiple 2D digital human actions of the second image in the target scene, and the multiple 2D digital human actions constitute the basic action library of the second image in the target scene. The 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library. For example, the reference action library includes the action of waving, and the basic action library also includes the action of waving.

[0054] In an optional manner, the starting and ending postures of multiple 2D digital human actions in the reference action library are the same. For example, the starting and ending postures of the 2D digital human action are both with the hands on the waist. The starting and ending postures of multiple 2D digital human actions in the basic action library are the same, and are the same as the starting postures of the 2D digital human actions in the reference action library, which can facilitate the connection of different 2D digital human actions.

[0055] In an optional manner, each 2D digital human action in the reference action library corresponds to an action type. The 2D digital human action A1 in the basic action library is obtained by migrating the 2D digital human action B1 in the reference action library. Then the action type of the 2D digital human action A1 is the action type of the 2D digital human action B1.

[0056] Step 303: output a basic action library, which is a 2D digital human action library of the second image in the target scene.

[0057] In this embodiment, the public cloud sends a basic action library to the terminal device, and the terminal device receives and stores the basic action library, which is a 2D digital human action library of the second image in the target scene.

[0058] In an optional manner, after generating a basic action library of the second image in the target scene, the basic action library can be used to generate a 2D digital human action video of the second image, and the process is as follows:

[0059] When generating a 2D digital human action video of the second image, the user operates the terminal device so that the terminal device displays an input interface, and the user operates in the input interface to input a driving voice and a second image file, and the driving voice is the voice in the finally generated 2D digital human action video. For example, in a speech scene, the driving voice is the speech content of the speaker. The second image file includes a 2D real-person photo of the second image or a 2D real-person action video of the second image. The 2D real-person photo is a full-body photo or a half-body photo, which matches the target scene. For example, the target scene is a standing speech scene, the 2D real-person photo is a full-body photo of the second image, the target scene is a sitting news broadcasting scene, the 2D real-person photo is a half-body photo of the second image, and the 2D real-person action video is a recorded video of the second image with action. The action in the 2D real-person action video may match the target scene or may not match the target scene. For example, the target scene is a standing speech scene, and the action may be a hand action, etc. The terminal device sends the driving voice and the second image file to the public cloud. The public cloud receives the driving voice and the second image file.

[0060] The public cloud obtains a voice-driven limb model, which is a deep learning model, such as a convolutional neural network model. The public cloud inputs the driving voice and the image file of the first image into the voice-driven limb model to obtain an output, which is a 2D digital human action video. In the disclosed embodiment, the specific structure of the voice-driven limb model is not limited, and any model that can generate a 2D digital human action video based on the driving voice and image file can be used as the voice-driven limb model.

[0061] After generating the 2D digital human action video, the public cloud sends the 2D digital human action video to the terminal device. The public cloud marks the action segments to be edited in the 2D digital human action video. The action segments to be edited are action segments in the 2D digital human action video that need to be adjusted. For example, the action segments that need to be adjusted are action segments with irregular or unsmooth movements.

[0062] Each 2D digital human action in the basic action library of the second image corresponds to an action type. The public cloud determines the 2D digital human action of the same action type as the action segment to be edited in the basic action library of the second image. For action segment A in the action segment to be edited, the 2D digital human action B in the basic action library has the same action type as action segment A. In the 2D digital human action video, the public cloud uses 2D digital human action B to replace action segment A. Since the starting posture and ending posture of 2D digital human action B and action segment A may be different, it is also necessary to adjust 2D digital human action B so that 2D digital human action B is inserted into the 2D digital human action video and the connection is smooth. In this way, after replacing each action segment in the action segment to be edited, the edited 2D digital human action video is obtained. In the edited 2D digital human action video, the actions are standardized and meet the requirements.

[0063] It should be noted that the driving voice can be a voice recorded by a real person of the second image, or it can be a voice not recorded by a real person of the second image. When the action in the 2D real person action video matches the target scene, the final generated 2D digital human action video is closer to the second image. The second image file is the same as the first image file, or different.

[0064] Optionally, the voice-driven limb models corresponding to 2D real-life photos and 2D real-life action videos are different. For 2D real-life photos, the voice-driven limb model is trained based on the 2D real-life photos. For 2D real-life action videos, the voice-driven limb model is trained based on the 2D real-life action videos. The training process is the same.

[0065] The training process of the voice-driven limb model of 2D real-life photos is as follows: obtain multiple video data and an initial voice-driven limb model, each action video is a video data with voice and action, and the voice and action match. For the first video data, input the voice and video image of each video frame into the initial voice-driven limb model, obtain the video frame with voice and action corresponding to each video frame, calculate the difference between the obtained video frame and the original video frame, obtain the loss corresponding to the first video data, and update the parameters in the initial voice-driven limb model based on the loss. Continue to update the parameters based on the second video data and the updated voice-driven limb model until the loss is less than a certain value when using a certain video data, or the number of training times reaches a certain value, and the voice-driven limb model is obtained.

[0066] The training process of the voice-driven limb model of 2D real-life action videos can be found in the above training process and will not be repeated here.

[0067] The training process here is only an example, and any method that can train a voice-driven body language model can be applied to the embodiments of the present disclosure.

[0068] The following describes the method flow of generating an action library by taking the first image file including a 2D real-life photo as an example. Figure 4 Steps 401 to 403.

[0069] Step 401, obtaining a reference action library of a first image in a target scene and a 2D real-life photo of a second image, wherein the reference action library includes multiple 2D digital human actions of the first image, and the starting postures and ending postures of the multiple 2D digital human actions are the same.

[0070] In this embodiment, the terminal device is provided with an input interface, and the user inputs the reference action library and the 2D real-life photo of the second image through the input interface. The reference action library includes common 2D digital human actions in the target scene. The 2D real-life photo is a full-body photo or a half-body photo, which matches the target scene. For example, if the target scene is a standing speech scene, the 2D real-life photo is a full-body photo of the second image, in which the person faces forward and stands naturally and relaxedly. If the target scene is a sitting speech scene, the 2D real-life photo is a half-body photo of the second image. The terminal device sends the reference action library and the 2D real-life photo of the second image to the public cloud. The public cloud receives the reference action library and the 2D real-life photo of the second image.

[0071] Alternatively, the terminal device is provided with an input interface, and the user inputs the 2D real-life photo of the second image and the scene identifier of the target scene through the input interface, and the terminal device sends the 2D real-life photo of the second image and the scene identifier of the target scene to the public cloud. The public cloud receives the 2D real-life photo of the second image and the scene identifier. The public cloud stores the correspondence between the scene identifier and the reference action library, and the public cloud determines the reference action library corresponding to the scene identifier of the target scene in the correspondence.

[0072] Step 402, based on the 2D real-life photo of the second image and the reference action library, generate a basic action library of the second image in the target scene, wherein the basic action library includes multiple 2D digital human actions of the second image, the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library, and the basic action library is used to generate a 2D digital human action video of the second image.

[0073] In this embodiment, the public cloud obtains a 2D real-person action generation model, and for a 2D digital human action A in a reference action library, the first video frame and the 2D real-person photo in the 2D digital human action A are input into the 2D real-person action generation model to obtain the first video frame in the 2D digital human action of the second image, and then the second video frame and the 2D real-person photo in the 2D digital human action A are input into the 2D real-person action generation model to obtain the second video frame in the 2D digital human action of the second image, and based on this method, the 2D digital human action of the second image in the target scene is obtained in sequence, and the 2D digital human action is the same as the 2D digital human action A. For example, the 2D digital human action A is an action of waving with the right hand in front, and the generated 2D digital human action is also an action of waving with the right hand in front. Other 2D digital human actions in the reference action library are similar to the 2D digital human action A, and can also correspond to the 2D digital human action of the second image in the target scene. Here, the 2D real-life motion generation model is taken as an example to input one 2D digital human motion at a time. In other optional methods, the 2D real-life motion generation model can also input multiple 2D digital human motions at the same time and output multiple 2D digital human motions of the second image in the target scene.

[0074] The 2D real-life action generation model is obtained through training. The training process is as follows: obtain multiple action videos and an initial 2D real-life action generation model, each action video is an image action video, input the i-th video frame containing the action and a video frame containing the image in the first action video into the initial 2D real-life action generation model, obtain an output video frame, calculate the difference between the output video frame and the i-th video frame, assuming that there are N video frames containing the action (N is greater than or equal to 1), then N differences can be calculated, and the loss is calculated based on the N differences (such as determining the variance of the N differences as the loss), and the initial 2D real-life action generation model is updated based on the loss. Continue to update the parameters based on the second video frame and the updated 2D real-life action generation model until the loss is less than a certain value, or the training reaches a certain number of times to obtain the 2D real-life action generation model.

[0075] Here, a single sample is used for each parameter update, and in some other optional methods, multiple samples may be used for each parameter update. In addition, this is only a possible training method, and the embodiments of the present disclosure do not limit this.

[0076] In an optional manner, before step 402, some pre-processing may be performed on the 2D real-life photo, such as cropping, so that the size of the 2D real-life photo meets the requirements.

[0077] Step 403: output a basic action library, where the basic action library is a 2D digital human action library of the second image in the target scene.

[0078] use Figure 4 The process shown can obtain a basic action library of the second image in the target scene, and subsequently use the basic action library to generate a 2D digital human action video of the second image in the target scene.

[0079] The following describes the method flow of generating an action library by taking the first image file including a 2D real-life action video as an example. Figure 5 Steps 501 to 505.

[0080] Step 501, obtaining a reference action library of a first image in a target scene and a 2D real-life action video of a second image, wherein the reference action library includes multiple 2D digital human actions of the first image, and the starting postures and ending postures of the multiple 2D digital human actions are the same.

[0081] In this embodiment, the terminal device is provided with an input interface, through which the user inputs a reference action library and a 2D real-person action video of the second image, and the reference action library includes common 2D digital human actions in the target scene. For example, the target scene is a short video live broadcast scene, and the reference action library includes common 2D digital human actions in the short video live broadcast scene. The 2D real-person action video matches the target scene. For example, the target scene is a standing speech scene, and the 2D real-person action video is a video action of the second image standing. For another example, the target scene is a short video live broadcast scene, and the 2D real-person action video includes a 2D real-person full-body or half-body action video with hand movements and limb movements. The terminal device sends the reference action library and the second image's 2D real-person action video to the public cloud. The public cloud receives the reference action library and the second image's 2D real-person action video.

[0082] Alternatively, the terminal device may provide an input interface, through which the user may input the 2D real-life action video of the second image and the scene identifier of the target scene, and the terminal device may send the 2D real-life action video of the second image and the scene identifier of the target scene to the public cloud. The public cloud receives the 2D real-life action video of the second image and the scene identifier. The public cloud stores the correspondence between the scene identifier and the reference action library, and the public cloud determines the reference action library corresponding to the scene identifier of the target scene in the correspondence.

[0083] Step 502, based on the 2D real-life action video of the second image and the reference action library, generate a basic action library of the second image in the target scene, wherein the basic action library includes multiple 2D digital human actions of the second image, the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library, and the basic action library is used to generate the 2D digital human action video of the second image.

[0084] In this embodiment, the public cloud obtains a 2D real-person action generation model, and obtains a video frame containing a second image in a 2D real-person action video. For a 2D digital human action A in a reference action library, the first video frame in the 2D digital human action A and the video frame containing the second image are input into the 2D real-person action generation model to obtain the first video frame in the 2D digital human action of the second image in the target scene, and then the second video frame in the 2D digital human action A and the video frame containing the second image are input into the 2D real-person action generation model to obtain the second video frame in the 2D digital human action of the second image in the target scene, and based on this method, the 2D digital human action of the second image in the target scene is obtained in sequence, and the 2D digital human action is the same as the 2D digital human action A. For example, the 2D digital human action A is a gesture of the left hand, and the 2D digital human action of the second image in the target scene is also a gesture of the left hand. Other 2D digital human actions in the reference action library are similar to the 2D digital human action A, and can also correspond to the 2D digital human action of the second image in the target scene. Here, the 2D real-life motion generation model is taken as an example to input one 2D digital human motion at a time. In other optional methods, the 2D real-life motion generation model can also input multiple 2D digital human motions at the same time and output multiple 2D digital human motions of the second image in the target scene.

[0085] In an optional manner, the 2D real-life action generation model is updated using a 2D real-life action video, and the second image can be fine-tuned. For example, in the 2D real-life action video, a video frame containing the second image and a single video frame containing the action of the second image are obtained, input into the 2D real-life action generation model, an output video frame is obtained, the loss between the output video frame and the single video frame is calculated, and based on the loss, the 2D real-life action generation model is updated. The process can be performed multiple times to update the 2D real-life action generation model. In this way, since the 2D real-life action generation model is updated using the 2D real-life action video of the second image, the 2D real-life action generation model can learn the characteristics of the second image when performing an action, so that the subsequent generated 2D digital human action of the second image is closer to the real person of the second image.

[0086] Step 503: Acquire the proprietary action information corresponding to the 2D real-life action video, wherein the proprietary action information includes the time period of the 2D digital human's proprietary action in the 2D real-life action video.

[0087] In this embodiment, in some scenarios, there is a proprietary action of the second image in the 2D real-life action video, and the user can input the time period of the proprietary action in the 2D real-life action video in the input interface of step 501, and the terminal device obtains the proprietary action information, which includes the time period of the proprietary action in the 2D real-life action video, and the terminal device sends the proprietary action information to the public cloud. The public cloud receives the proprietary action information sent by the terminal device.

[0088] In an optional manner, the public cloud provides a proprietary action annotation interface to the user through a terminal device, and the proprietary action annotation interface is provided with a start selection option and an end selection option of the proprietary action. The user can select the start time point of the proprietary action through the start selection option, and can select the end time point of the proprietary action through the end selection option. The start time point and the end time point constitute the time period of the proprietary action. The user can select all proprietary actions in this manner. After the user completes the selection, the terminal device obtains the proprietary action information and sends the start time point and the end time point to the public cloud. The public cloud receives the proprietary action information sent by the terminal device.

[0089] When there are multiple unique actions, the user can also input the action type of the unique action to distinguish each unique action, for example, the action type is making a fist with both hands, etc. The unique action information can be expressed as {"unique action":"making a fist with both hands","time period":[18.4,29.1]}.

[0090] Step 504: Generate a proprietary action library corresponding to the 2D real-life action video based on the 2D real-life action video and the proprietary action information.

[0091] In this embodiment, the public cloud uses the time period of the proprietary action in the 2D real-life action video to capture the video of the proprietary action in the 2D real-life action video. For each proprietary action video, the video of the proprietary action is input into the proprietary action generation model to obtain the 2D digital human proprietary action corresponding to the proprietary action. The starting posture and ending posture of each 2D digital human proprietary action in the proprietary action library are the same, and the 2D digital human proprietary action in the proprietary action library is the same as the starting posture of the 2D digital human action in the basic action library.

[0092] In an optional manner, the proprietary action generation model and the 2D digital human generation model are both action generation models, and the feature extraction parts of the two are the same, so the proprietary action generation model can borrow a part of the 2D digital human generation model, and this part does not need to be trained. Figure 5 Indicated by dashed arrows.

[0093] In another optional way, the proprietary action generation model is trained independently, and the public cloud obtains multiple samples, each sample includes a 2D real-person action and a 2D digital human action corresponding to the 2D real-person action, and the starting posture and ending posture of the 2D digital human action are the same. The training process is: input the 2D real-person action in the first sample into the initial proprietary action generation model to obtain the 2D digital human action, calculate the loss of the 2D digital human action and the 2D digital human action in the first sample, and update the parameters of the initial proprietary action generation model based on the loss. Input the 2D real-person action in the second sample into the updated proprietary action generation model to obtain the 2D digital human action, and calculate the loss of the 2D digital human action and the 2D digital human action in the second sample. If the loss is greater than a certain value, continue to update the parameters and train until the loss is less than a certain value to obtain a proprietary action generation model. The proprietary action model can convert 2D real-person actions into corresponding 2D digital human actions.

[0094] Step 505: output the basic action library and the proprietary action library, which constitute the 2D digital human action library of the second image in the target scene.

[0095] In this embodiment, the public cloud sends the basic action library and the proprietary action library to the terminal device, and the terminal device receives and stores the basic action library and the proprietary action library.

[0096] use Figure 5 The process shown can obtain the basic action library and the proprietary action library of the second image in the target scene, and subsequently use the basic action library and the proprietary action library to generate a 2D digital human action video of the second image in the target scene.

[0097] In the disclosed embodiment, given a reference action library and an image file of a new image, the 2D digital human actions in the reference action library can be quickly migrated to the newly added image. The newly added image does not need to re-record the action library, which solves the problem that the 2D digital human needs to shoot a large number of actions with real people, and can improve the efficiency of generating the action library.

[0098] In addition, based on the 2D digital human motion generation model, it is ensured that the 2D digital human motion of each image meets the standards and remains consistent. At the same time, because the action video and proprietary action annotations of the new image are used, the naturalness and uniqueness of the new image's motion are guaranteed, solving the problem of non-standard and unnatural movements of different images.

[0099] In addition, only action migration is performed, which does not require a large amount of hardware and equipment support, thus saving costs.

[0100] The following describes the device provided by the embodiment of the present disclosure. Figure 6The structure diagram of the device for generating the 2D digital human action library provided by the embodiment of the present disclosure is shown in FIG. The device can be implemented as part or all of the device through software, hardware, or a combination of both. The device provided by the embodiment of the present disclosure can implement the embodiment of the present disclosure. Figures 3 to 5 According to the process shown, the device includes: an acquisition module 610 and a generation module 620, wherein:

[0101] The acquisition module 610 is used to acquire a first image file and a reference action library of the first image in a target scene, wherein the first image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image, and the reference action library includes multiple 2D digital human actions of the first image, and can be specifically used to execute the acquisition function of the above step 301, as well as the implicit steps included in step 301;

[0102] The generation module 620 is used to generate a basic action library of the second image in the target scene based on the first image file and the reference action library, wherein the basic action library includes multiple 2D digital human actions of the second image, and the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library. The basic action library is used to generate a 2D digital human action video of the second image, and can specifically be used to execute the generation function of the above-mentioned step 302, as well as the implicit steps contained in step 302.

[0103] In an optional manner, the generating module 620 is used to:

[0104] The first image file and the 2D digital human actions in the reference action library are input into a 2D real-life action generation model to generate multiple 2D digital human actions of the second image, wherein the multiple 2D digital human actions of the second image constitute the basic action library, and the 2D real-life action generation model is obtained based on training of multiple action videos.

[0105] In an optional manner, the first image file includes a live-action video of the second image;

[0106] The generating module 620 is further used for:

[0107] Before inputting the first image file and the 2D digital human actions in the reference action library into the 2D real-person action generation model, the 2D real-person action video is used to update the 2D real-person action generation model.

[0108] In an optional manner, the first image file includes a 2D live-action video of the second image;

[0109] The acquisition module 610 is further configured to acquire the proprietary action information corresponding to the 2D live-action video, wherein the proprietary action information includes a time period of the proprietary action in the 2D live-action video;

[0110] The generating module 620 is further used to generate a proprietary action library of the second image in the target scene based on the 2D real-life action video and the proprietary action information, wherein the proprietary action library includes the 2D digital human proprietary actions of the second image.

[0111] In an optional manner, the acquisition module 610 is further configured to:

[0112] Providing a proprietary action annotation interface, wherein the proprietary action annotation interface is used for a user to annotate the proprietary action of the 2D real-life action video;

[0113] The proprietary action information fed back by the user through the proprietary action annotation interface is obtained.

[0114] In an optional manner, the acquisition module 610 is used to:

[0115] receiving the reference action library input by a user; or,

[0116] A scene identifier of the target scene input by a user is received, and the reference action library corresponding to the scene identifier is determined.

[0117] In an optional manner, the acquisition module 610 is further used to acquire a driving voice and a second image file after generating a basic action library of the second image in the target scene, wherein the second image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image;

[0118] The generating module 620 is further used to generate a 2D digital human action video matching the driving voice based on the driving voice and the second image file;

[0119] The acquisition module 610 is further used to acquire the action segment to be edited of the 2D digital human action video;

[0120] The generating module 620 is further configured to use the basic action library to edit the action segment to be edited in the 2D digital human action video to obtain the edited 2D digital human action video.

[0121] Wherein, both the acquisition module 610 and the generation module 620 can be implemented by software or by hardware. Exemplarily, the generation module 620 is taken as an example to introduce the implementation of the generation module 620. Similarly, the implementation of the acquisition module 610 can refer to the implementation of the generation module 620.

[0122] As an example of a software functional unit, the generation module 620 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the generation module 620 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code may be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code may be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region may include multiple AZs.

[0123] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0124] As an example of a hardware functional unit, the generation module 620 may include at least one computing device, such as a server, etc. Alternatively, the generation module 620 may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0125] The multiple computing devices included in the generation module 620 can be distributed in the same region or in different regions. The multiple computing devices included in the generation module 620 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the generation module 620 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0126] It should be noted that, in other embodiments, the generation module 620 can be used to execute any step in the method for generating a 2D digital human action library, and the acquisition module 610 can be used to execute any step in the method for generating a 2D digital human action library.

[0127] The present disclosure also provides a computing device 700. Figure 7 As shown, computing device 700 includes: bus 702, processor 704, memory 706 and communication interface 708. Processor 704, memory 706 and communication interface 708 communicate through bus 702. Computing device 700 can be a server or a terminal device. It should be understood that the present disclosure does not limit the number of processors and memories in computing device 700.

[0128] The bus 702 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7 The bus 702 may include a path for transmitting information between various components of the computing device 700 (eg, the memory 706, the processor 704, and the communication interface 708).

[0129] The processor 704 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0130] The memory 706 may include a volatile memory, such as a random access memory (RAM). The memory 706 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0131] The memory 706 stores executable program codes, and the processor 704 executes the executable program codes to respectively implement the functions of the acquisition module 610 and the generation module 620, thereby implementing the method for generating a 2D digital human action library. That is, the memory 706 stores instructions for executing the method for generating a 2D digital human action library.

[0132] The present disclosure also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device may be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device may also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0133] like Figure 8 As shown, the computing device cluster includes at least one computing device 700. The memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the method for generating a 2D digital human action library.

[0134] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the method for generating the 2D digital human action library. In other words, the combination of one or more computing devices 700 may jointly execute the instructions for executing the method for generating the 2D digital human action library.

[0135] It should be noted that the memory 706 in different computing devices 700 in the computing device cluster can store different instructions, which are respectively used to execute part of the functions of the device for generating the image action library. That is, the instructions stored in the memory 706 in different computing devices 700 can implement the functions of one or more modules in the acquisition module 610 and the generation module 620.

[0136] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig. 9A possible implementation is shown. Fig. 9 As shown, the two computing devices include a first computing device 700A and a second computing device 700B, and the first computing device 700A and the second computing device 700B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory 706 in the first computing device 700A stores instructions for executing the functions of the generation module 620. At the same time, the memory 706 in the second computing device 700B stores instructions for executing the functions of the acquisition module 610.

[0137] It should be understood that Fig. 9 The functions of the first computing device 700A shown in FIG. 7 may also be completed by multiple computing devices 700. Similarly, the functions of the second computing device 700B may also be completed by multiple computing devices 700.

[0138] The present disclosure also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Figure 8 and Fig. 9 The connection mode of the computing device cluster is different in that the memory 706 in one or more computing devices 700 in the computing device cluster may store the same instructions for executing the method for generating the 2D digital human action library.

[0139] In some possible implementations, the memory 706 of one or more computing devices 700 in the computing device cluster may also store partial instructions for executing the method for generating the 2D digital human action library. In other words, the combination of one or more computing devices 700 may jointly execute the instructions for executing the method for generating the 2D digital human action library.

[0140] The present disclosure also provides a computer program product including instructions. The computer program product may be software or a program product including instructions that can be run on a computing device or stored in any available medium. When the computer program product is run on at least one computing device, the at least one computing device executes a method for generating a 2D digital human action library.

[0141] The present disclosure also provides a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be stored by a computing device or a data storage device such as a data center that includes one or more available media. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk). The computer-readable storage medium includes instructions that instruct the computing device to execute a method for generating a 2D digital human action library.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present disclosure, rather than to limit them. Although the present disclosure has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present disclosure.

Claims

1. A method for generating a two-dimensional 2D digital human action library, characterized in that: The method comprises: Acquire a first image file and a reference action library of the first image in a target scene, wherein the first image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image, and the reference action library includes a plurality of 2D digital human actions of the first image; Based on the first image file and the reference action library, a basic action library of the second image in the target scene is generated, wherein the basic action library includes multiple 2D digital human actions of the second image, the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library, and the basic action library is used to generate a 2D digital human action video of the second image.

2. The method according to claim 1, characterized in that The step of generating a basic action library of the second image in the target scene based on the first image file and the reference action library comprises: The first image file and the 2D digital human actions in the reference action library are input into a 2D real-life action generation model to generate multiple 2D digital human actions of the second image, wherein the multiple 2D digital human actions of the second image constitute the basic action library, and the 2D real-life action generation model is obtained based on training of multiple action videos.

3. The method according to claim 2, characterized in that The first image file includes a live-action video of the second image; Before inputting the first image file and the 2D digital human motion in the reference motion library into the 2D real person motion generation model, the method further includes: Using the 2D real-life action video, the 2D real-life action generation model is updated.

4. The method according to any one of claims 1 to 3, characterized in that: The first image file includes a 2D live-action video of the second image; The method further comprises: Acquire proprietary action information corresponding to the 2D live-action video, wherein the proprietary action information includes a time period of the proprietary action in the 2D live-action video; Based on the 2D real-life action video and the proprietary action information, a proprietary action library of the second image in the target scene is generated, wherein the proprietary action library includes the 2D digital human proprietary actions of the second image.

5. The method according to claim 4, characterized in that The obtaining of the proprietary action information corresponding to the 2D real-life action video includes: Providing a proprietary action annotation interface, wherein the proprietary action annotation interface is used for a user to annotate the proprietary action of the 2D real-life action video; The proprietary action information fed back by the user through the proprietary action annotation interface is obtained.

6. The method according to any one of claims 1 to 5, characterized in that: Obtaining the reference action library includes: receiving the reference action library input by a user; or, A scene identifier of the target scene input by a user is received, and the reference action library corresponding to the scene identifier is determined.

7. The method according to any one of claims 1 to 6, characterized in that: After generating a basic action library of the second image in the target scene, the method further includes: Acquire a driving voice and a second image file, wherein the second image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image; Based on the driving voice and the second image file, generating a 2D digital human action video matching the driving voice; Obtaining the action segment to be edited of the 2D digital human action video; The basic action library is used to edit the action segment to be edited in the 2D digital human action video to obtain the edited 2D digital human action video.

8. A device for generating a 2D digital human action library, characterized in that: The device comprises: An acquisition module, used to acquire a first image file and a reference action library of the first image in a target scene, wherein the first image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image, and the reference action library includes a plurality of 2D digital human actions of the first image; A generation module is used to generate a basic action library of the second image in the target scene based on the first image file and the reference action library, wherein the basic action library includes multiple 2D digital human actions of the second image, the 2D digital human actions in the basic action library are the same as the 2D digital human actions in the reference action library, and the basic action library is used to generate a 2D digital human action video of the second image.

9. The device according to claim 8, characterized in that The generating module is used for: The first image file and the 2D digital human actions in the reference action library are input into a 2D real-life action generation model to generate multiple 2D digital human actions of the second image, wherein the multiple 2D digital human actions of the second image constitute the basic action library, and the 2D real-life action generation model is obtained based on training of multiple action videos.

10. The device according to claim 9, characterized in that The first image file includes a live-action video of the second image; The generating module is further used for: Before inputting the first image file and the 2D digital human actions in the reference action library into the 2D real-person action generation model, the 2D real-person action video is used to update the 2D real-person action generation model.

11. The device according to any one of claims 8 to 10, characterized in that The first image file includes a 2D live-action video of the second image; The acquisition module is further used to acquire the proprietary action information corresponding to the 2D live-action video, wherein the proprietary action information includes a time period of the proprietary action in the 2D live-action video; The generation module is further used to generate a proprietary action library of the second image in the target scene based on the 2D real-life action video and the proprietary action information, wherein the proprietary action library includes the 2D digital human proprietary actions of the second image.

12. The device according to claim 11, characterized in that The acquisition module is further used for: Providing a proprietary action annotation interface, wherein the proprietary action annotation interface is used for a user to annotate the proprietary action of the 2D real-life action video; The proprietary action information fed back by the user through the proprietary action annotation interface is obtained.

13. The device according to any one of claims 8 to 12, characterized in that The acquisition module is used to: receiving the reference action library input by a user; or, A scene identifier of the target scene input by a user is received, and the reference action library corresponding to the scene identifier is determined.

14. The device according to any one of claims 8 to 13, characterized in that The acquisition module is further used to acquire a driving voice and a second image file after generating a basic action library of the second image in the target scene, wherein the second image file includes a 2D real-life photo of the second image or a 2D real-life action video of the second image; The generating module is further used to generate a 2D digital human action video matching the driving voice based on the driving voice and the second image file; The acquisition module is further used to acquire the action segment to be edited of the 2D digital human action video; The generating module is further used to use the basic action library to edit the action segment to be edited in the 2D digital human action video to obtain the edited 2D digital human action video.

15. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is configured to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 7.

16. A computer-readable storage medium, characterized in that: The method comprises computer program instructions. When the computer program instructions are executed by a computing device cluster, the computing device cluster performs the method according to any one of claims 1 to 7.

Citation Information

Cited By

  • Digital human generation and interaction method, equipment and medium

    CN121547664A