Method and device for generating digital human synthesis video

Abstract

The invention relates to a digital human synthesis method and device, electronic equipment and a storage medium, and relates to the field of digital human synthesis. The method comprises the following steps: determining a virtual image of the digital human according to configuration information of the digital human; obtaining the text content of the poked manuscript of the digital person; determining a matched three-dimensional scene based on the text content; determining expression information of the digital human based on the virtual image, the text content and the three-dimensional scene; and based on the expression information, executing a digital human synthesis task to obtain a synthetic video of the digital human. By applying the method and the device, the digital person can be directly configured, the digital person is combined with the three-dimensional scene, and the port broadcast manuscript is configured for the digital person in the three-dimensional scene, so that the configuration of the digital person and the post-processing of the synthesized video can be normalized into one step for processing, and the production efficiency of the digital person synthesized video is improved.

Description

Technical Field

[0001] The present application relates to the technical field of three-dimensional graphics and image processing and virtual digital human synthesis, and in particular to a method and device for generating digital human synthesized video. Background Art

[0002] Virtual digital humans are virtual images with appearance characteristics and behavior patterns created through technological means such as modeling, motion capture or AI, and presented through display devices. Virtual digital humans can be applied to many fields such as entertainment, finance, and medical care, such as playing a role in a movie, serving as a virtual account manager of a bank, or serving as a virtual assistant. The value created by virtual humans is mainly to break the limitations of physical space and provide more immersion, participation, and interaction. In the existing digital human synthesis scheme, it is generally necessary to purchase synthesis services through the digital human synthesis platform, download the digital human synthesis video after configuration operations, and then further process the synthesized video. In this way, the various links of the digital human synthesis scheme are scattered and cumbersome, and it is impossible to directly perform video post-production, resulting in low efficiency in the production of digital human synthesis videos. Summary of the invention

[0003] The embodiments of the present disclosure provide a method, device, electronic device and storage medium for generating digital human synthesized videos. The digital human is directly edited by embedding a plug-in in the software, and the digital human is combined with a three-dimensional scene, so that the digital human synthesized video can be directly post-produced, thereby effectively improving the production efficiency of the digital human synthesized video.

[0004] In a first aspect, an embodiment of the present disclosure provides a method for generating a digital human synthesized video, comprising: determining a virtual image of the digital human based on configuration information of the digital human; obtaining text content of a spoken script of the digital human; determining a matching three-dimensional scene based on the text content; determining expression information of the digital human based on the virtual image, text content and three-dimensional scene; and executing a digital human synthesis task based on the expression information to obtain a synthesized video of the digital human.

[0005] In a second aspect, an embodiment of the present disclosure provides a device for generating a digital human synthesized video, comprising: an image determination unit, configured to determine a virtual image of a digital human according to configuration information of the digital human; a text acquisition unit, configured to acquire text content of a spoken script of the digital human; a scene determination unit, configured to determine a matching three-dimensional scene based on the text content; an expression determination unit, configured to determine expression information of the digital human based on the virtual image, text content and three-dimensional scene; and a digital human synthesis unit, configured to perform a digital human synthesis task based on the expression information to obtain a synthesized video of the digital human.

[0006] By applying the technical solution disclosed in the present invention, the digital human can be directly configured and combined with the three-dimensional scene. By configuring the digital human with a spoken script in the three-dimensional scene, the configuration of the digital human and the post-processing of the synthesized video can be integrated into one step, thereby improving the production efficiency of the digital human synthesized video.

[0007] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0009] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for generating a digital human synthetic video disclosed in the present invention;

[0010] Figure 2 A schematic diagram of an application scenario of the method for generating digital human synthetic video disclosed in the present invention;

[0011] Figure 3 A flowchart of another embodiment of the method for generating a digital human synthetic video disclosed in the present invention;

[0012] Figure 4 A schematic diagram of the structure of an embodiment of a device for generating digital human synthetic video disclosed in the present invention;

[0013] Figure 5 A schematic diagram of the structure of an embodiment of an electronic device disclosed herein. DETAILED DESCRIPTION

[0014] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present disclosure. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present disclosure belongs.

[0015] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present disclosure. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprising" and / or "including" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0016] In the absence of conflict, the embodiments in the present disclosure and the features in the embodiments may be combined with each other.

[0017] In order to make the technical solutions and advantages of the present disclosure more clearly understood, the present disclosure is further described in detail below in conjunction with the accompanying drawings and specific embodiments.

[0018] Figure 1 FIG. 1 shows a process 100 of an embodiment of a method for generating a digital human synthetic video of the present disclosure. Figure 1 As shown, the digital human synthesis method of this embodiment may include the following steps:

[0019] Step 101, determining the virtual image of the digital human according to the configuration information of the digital human.

[0020] In this embodiment, the configuration information of the digital human may be obtained first. The configuration information may be set by a technician according to a pre-established three-dimensional scene or according to the content of a spoken script, or according to a specific character image. The three-dimensional scene may include multiple three-dimensional elements, such as houses, walls, roads, etc.

[0021] The configuration information may include the image, gender, clothing, display area, etc. of the digital person. The image may include a person, a cartoon, etc. Furthermore, if it is a person image, the configuration information may further include the person's clothing, hairstyle, makeup, etc. The display area may include the whole body, half body, etc.

[0022] After receiving the above configuration information, the virtual image of the digital person can be determined. Specifically, a pre-set virtual image set can be queried to filter out virtual images with keywords related to the configuration information. Specifically, the above virtual image set includes multiple virtual images, and each virtual image can correspond to multiple keywords. The keywords in the configuration information can be matched with the keywords of each virtual image.

[0023] Step 102, obtaining the text content of the digital human's spoken script.

[0024] In this embodiment, the digital human can serve as a virtual host. In this scenario, the text content of the digital human's oral script can be first obtained. The oral script here can be the content that the virtual host needs to narrate. The above text content can be adapted to a pre-established three-dimensional scene. For example, the text content includes content that introduces a three-dimensional model in a three-dimensional scene. Specifically, the text content of the above oral script can be obtained through a pre-set input box. Alternatively, the text content of the oral script can be generated by artificial intelligence.

[0025] Step 103: Determine a matching three-dimensional scene based on the text content.

[0026] After obtaining the text content, the text content can be analyzed to determine the matching 3D scene. Specifically, keywords can be extracted from the text content. Then, 3D materials matching the keywords are searched from a preset 3D material resource library. A 3D scene is built based on the 3D materials. Alternatively, the text content can be compared with a preset template to determine the feature words in the text content. The 3D material is determined based on the feature words. Based on the above 3D materials and the preset 3D scene template, a matching 3D scene is determined.

[0027] Step 104, based on the virtual image, text content and three-dimensional scene, determine the expression information of the virtual image.

[0028] After determining the virtual image, text content and three-dimensional scene, the expression information of the virtual image can be determined. Here, the expression information may include: the position, action, expression, etc. of the digital person. Specifically, if the text content contains an introduction to a specific three-dimensional model, the expression information may include an action for indicating the three-dimensional model. If the three-dimensional scene includes a specific three-dimensional material, the position of the digital person may be the same as the position of the three-dimensional material. For example, if the three-dimensional scene includes a weather element, the position of the digital person may be set at a specified relative position of the weather element. Alternatively, insert a nodding action every few sentences in the text content. It is understandable that the expression information may include multiple actions or expressions corresponding to different time points.

[0029] Step 105, based on the expression information, execute the digital human synthesis task to obtain a synthesized video of the digital human.

[0030] After determining the above expression information, the digital human synthesis task can be performed. Specifically, the above expression information can be displayed on the basis of the digital human at the corresponding time point to obtain a green background video of the digital human. Then, the green background video of the digital human is used as an editing element and combined with the three-dimensional scene to obtain a synthesized video of the digital human. After the synthesis is completed, a synthesized video of the digital human can be obtained. The synthesized video can include a three-dimensional scene, and the digital human speaks in the three-dimensional scene and expresses some actions and expressions during the speaking process.

[0031] In some specific practices, the process of obtaining the green background video of the digital human can be called the digital human generation service. The digital human generation service can be embedded in the existing graphic quick editing packaging tool in the form of a plug-in, so that the digital human synthesis task can be completed directly through the graphic quick editing packaging tool. After the configuration of the digital human and the three-dimensional scene is completed, the materials between the start time and the end time of the video can be packaged and synthesized to obtain a synthesized video of the digital human. It can be understood that the synthesized video is a three-dimensional video. At the same time, the engineering file of the synthesized video can also be obtained. The engineering file can be a file of a specified format, and the engineering file can be opened and edited by the graphic quick editing packaging tool.

[0032] Figure 2 A schematic diagram showing an application scenario of the method for generating digital human synthetic video disclosed in the present invention. Figure 2 In the application scenario shown, a graphic quick-editing packaging tool is installed on the terminal device, and a plug-in for the digital human generation service is embedded in the graphic quick-editing packaging tool. The user first creates a new project through the graphic quick-editing packaging tool and sets key parameters such as composition, resolution and frame rate. By opening the digital human generation service plug-in in the UI interface, specific operations for synthesizing the virtual image are performed. The format of the digital human is unified with the project and there is no need to reset it. Users can enter the configuration information of the digital human in the digital human generation service plug-in. The above configuration information may include: character, female, full body. Further, it may include keywords such as "teacher", "teaching aids", etc. The above configuration information is used to determine the following Figure 2 The virtual image shown. Then, the content of the class lecture can be input as the text content of the oral manuscript. By analyzing the above text content, various types of materials required can be determined, including three-dimensional materials and two-dimensional materials. Two-dimensional materials can include text, sound, pictures, videos, and belong to image-related resources. Three-dimensional materials can include three-dimensional scenes, models, animations, effects, etc., and belong to graphic-related resources. The above materials can be used to build a three-dimensional scene of the classroom. The three-dimensional scene includes three-dimensional models such as blackboards and globes. The graphic quick editing packaging tool analyzes the above virtual image, text content and three-dimensional scene to determine the actions and expressions that the digital person as a teacher needs to have when teaching. For example, when talking about the ocean part on the earth, it is necessary to use teaching aids to indicate the ocean on the earth. In this way, the actions indicated by the teaching aids can be inserted into the teaching manuscript. After the setting is completed, the actions of the digital person can be matched with the foreground animation of the three-dimensional scene to obtain a synthesized video. The overall setting can be output for preview, and the synthesized video can be modified during the preview process. Then, according to the above actions and expressions, the digital person is synthesized to finally obtain a video of the virtual teacher's lecture.

[0033] The method for generating digital human synthetic videos provided by the above-mentioned embodiments of the present disclosure can directly configure the image of the digital human and the spoken script, and combine the green background video of the digital human with the three-dimensional scene, so as to realize the integration of the configuration of the digital human and the post-processing of the synthetic video into one step, thereby improving the production efficiency of the digital human synthetic videos.

[0034] Continue to see Figure 3 , which shows a process 300 of another embodiment of the method for generating a digital human synthetic video according to the present disclosure. Figure 3 As shown, the method in this embodiment may include the following steps:

[0035] Step 301, determining the virtual image of the digital human according to the configuration information of the digital human.

[0036] Step 302, obtaining the text content of the digital human's spoken script.

[0037] Step 303, extracting keywords from the text content; determining matching 3D materials based on the keywords and a preset 3D material resource library; and determining matching 3D scenes based on the 3D materials.

[0038] In this embodiment, keywords can be extracted from the text content. Specifically, the text content can be processed using a natural language processing algorithm to obtain words of a specified type as keywords. For example, the extracted nouns can be used as keywords. Alternatively, the text content can be compared with a preset template, and the words in the text content that are different from the template can be used as keywords. Alternatively, words used to represent a specified meaning can be used as keywords. For example, words representing weather can be used as keywords.

[0039] After obtaining the keyword, the 3D material matching the keyword can be selected from the preset 3D material resource library. Specifically, each 3D material in the 3D material resource library can correspond to an abstract or logo, and the similarity between the keyword and the abstract or logo is calculated. The 3D material with a similarity greater than a preset threshold is used as the 3D material matching the keyword.

[0040] Then, the obtained three-dimensional material can be used to determine the matching three-dimensional scene. Specifically, the position of the three-dimensional material in the three-dimensional scene can be determined according to the summary or identification of the three-dimensional material. For example, if the summary of the three-dimensional material is "sky", the three-dimensional material can be placed above the three-dimensional scene. Alternatively, if the summary of the three-dimensional material is "ground", the three-dimensional material can be placed below the three-dimensional scene. Alternatively, the obtained three-dimensional material can be combined with a preset three-dimensional scene template to obtain a matching three-dimensional scene. Specifically, the preset three-dimensional scene template can include fixed three-dimensional materials and replaceable three-dimensional materials. The obtained three-dimensional material can be placed at the position of the above-mentioned replaceable three-dimensional material to obtain a matching three-dimensional scene.

[0041] In some optional implementations of the present embodiment, the three-dimensional scene can also be determined according to the ratio of the text content and the preset composite video. In this implementation, the ratio of the composite video can include a horizontal screen of 16:9 and a vertical screen of 9:16. In order to present a better display effect, different rendering viewports can be set for different video ratios. In this way, different scenes can be better matched for different video ratios, so that the three-dimensional elements in the scene are more matched with the size of the rendering viewport.

[0042] Step 304, determining an appropriate voice tone according to the virtual image; determining an appropriate speech rate according to the text content; and determining an action of the virtual image according to the text content and / or the three-dimensional material in the three-dimensional scene.

[0043] In this embodiment, the appropriate voice can also be determined according to the virtual image. For example, if the virtual image is female, the female voice can be adapted. If the virtual image is male, the male voice can be adapted. Further, after the appropriate voice is determined, an audition entrance for the above-mentioned appropriate voice can be provided. If modification information for the appropriate voice is received, the appropriate voice can be adjusted according to the modification information.

[0044] The adaptive speech rate can be further determined based on the text content. Specifically, the adaptive speech rate can be determined based on the type of text content. For example, when the text content is a news release, the adaptive speech rate is N words per minute. Or, when the text content is a recitation, the adaptive speech rate is M words per minute. The adaptive speech rate can also be determined based on the number of words included in the text content. For example, when the number of words is greater than a preset threshold, the speech rate is K words per minute.

[0045] Furthermore, the action of the virtual image can also be determined based on the text content and / or the three-dimensional material in the three-dimensional scene. Specifically, when the three-dimensional scene includes a specific three-dimensional model and the text content also includes a description of the three-dimensional model, an action of introducing the three-dimensional model can be included. Alternatively, when the text content includes words for a certain direction, an action pointing to the above direction can be included. The action of the virtual image can be determined based on the number of paragraphs included in the text content and the number of three-dimensional models included in the three-dimensional scene. For example, an entry action can be set before the broadcast of the oral script, and an exit action can be set after the oral script is broadcast. In the oral script, micro-actions can also be set for the digital human every preset number of sentences, such as slight swings, to increase the simulation effect.

[0046] In some optional implementations of this embodiment, the sideways angle of the digital human can also be determined according to the three-dimensional scene. Specifically, the sideways angle of the digital human can be determined according to the three-dimensional material included in the three-dimensional scene. For example, when the three-dimensional material includes a certain type of material (such as a two-dimensional video), the sideways angle of the digital human can be set to an angle of 0 to 30 degrees toward the material. In some specific practices, setting information for the sideways angle of the digital human can also be received. And the sideways angle of the digital human is set according to the above setting information. It can be understood that the setting of the sideways angle of the digital human may not be limited to the three-dimensional scene, but may also be based on the oral content or the position of the virtual lens, etc.

[0047] Step 305: Determine the expression information of the digital human based on the virtual image, text content and three-dimensional scene.

[0048] Step 306, determining the green background video of the digital human according to the expression information; determining the key frame picture according to the green background video, the three-dimensional scene and the camera movement information; rendering according to the key frame picture to obtain the composite video.

[0049] In this embodiment, after determining the expression information of the digital human, a green background video of the digital human can be generated first. Then, the green background video is combined with the three-dimensional scene, and the key frame picture is determined in combination with the camera movement information. Specifically, the green background video can be matted, and then settings such as occlusion or position movement or scaling can be performed. Then the digital human is set at a specified position in the three-dimensional scene. The camera movement information here can include information such as the position and focal length of the virtual camera. Specifically, the picture where the action in the expression information changes and the picture where the material in the three-dimensional scene changes can be used as the key frame picture, and the picture where the camera movement information changes can also be used as the key frame picture. Alternatively, a frame can be taken as the key frame picture at every preset time interval. After obtaining the key frame picture, it can be rendered using an image rendering engine to obtain a composite video. Specifically, the key frame can be interpolated and then rendered to obtain a composite video.

[0050] In some optional implementations of this embodiment, when executing the synthesis task of a digital human, it is executed in the background and the execution progress of the synthesis task is displayed. In this way, it is convenient for the user to perform other operations during the execution of the synthesis task, thereby improving the operation experience.

[0051] In some specific practices, the above-mentioned three-dimensional materials, key frame images and camera movement information can all be displayed on the time code track.

[0052] Step 307, generating a subtitle file according to the text content and the adapted speech rate; and displaying the subtitles corresponding to the text content in the synthesized video.

[0053] In this embodiment, the text content and the adaptive speech speed can also be used to generate a subtitle file. It is understandable that the time code of the subtitle file is consistent with the oral script of the digital person, which can ensure the validity of the subtitle file and improve the production efficiency. The above-mentioned subtitle file can be a .srt file. The above-mentioned subtitle file can be exported separately for further processing. It is understandable that the subtitle file should be displayed corresponding to the oral content of the digital person. In addition, corresponding to videos of different proportions, the font, size and display length of the text in the subtitle file can be adaptively adjusted. In some specific practices, the above-mentioned subtitle file can support "one-click track", that is, by dragging the subtitle file or clicking to import the subtitle file, the subtitles can be superimposed on the UI layer. At the same time, the font size or line break in the subtitle file can be dynamically adaptively adjusted according to the rendering viewport or video ratio.

[0054] In some optional implementations of this embodiment, in response to receiving adjustment information for at least one of the digital human, the three-dimensional scene, and the text content, real-time rendering is performed according to the adjustment information.

[0055] If at least one adjustment information of the characters, 3D scenes and text content is received, real-time rendering is performed according to the adjustment information. In this way, the user can view the adjustment results in real time through the application interface. After the adjustment is completed, the time code pointer can be reset to the starting position 00 (hour): 00 (minute): 00 (second): 00 (frame) on the time code track, and the overall synthesis effect can be previewed in real time by clicking Play (or the shortcut key Space).

[0056] In this embodiment, after synthesizing the digital human through the above-mentioned graphic quick editing packaging tool, a synthetic video engineering file can be obtained. In the synthetic video engineering file, the digital human, subtitles, video and the three-dimensional model in the three-dimensional scene can be edited as an element. For example, the position, size and direction of each element can be adjusted by dragging. The time code track can be adjusted to achieve the appearance time and appearance mode of each element. By operating the synthetic video engineering file, a video file, subtitle file, etc. in a specified format can be obtained. The above-mentioned video file can include a synthetic video and a green background video of the digital human.

[0057] When you open a historical composite video project file, you can directly adjust the elements in it.

[0058] In some optional implementations of this embodiment, for a specific type of synthetic video engineering file, the synthetic video engineering file can also be updated by the text content of the updated oral script. Specifically, the file content can be subjected to text analysis to determine the keywords contained therein. The three-dimensional model in the synthetic video is replaced by the three-dimensional model corresponding to the above-mentioned keywords. For example, for a synthetic video of the weather forecast type, the correspondence between the keywords and the three-dimensional model can be pre-set, and the template of the three-dimensional model can be set at the same time. The above-mentioned template may include a fixed part and a variable part. When new text content is obtained, the variable part in the template can be updated by using the three-dimensional model corresponding to the keywords in the text content. The above-mentioned variable part may include but is not limited to: meteorological symbols, temperature values, wind force values, etc. When updating, the time code information of the three-dimensional model can be set with reference to the time code information of the subtitle file corresponding to the text content. In this way, the automated production of synthetic content of digital humans can be achieved.

[0059] In some optional implementations of this embodiment, if operation information on a historical composite video project file is received, the material in the historical composite video project file is added to a new time code track. In this implementation, the user can open the composite video by clicking or dragging the historical composite video project file, and at this time, the material in the composite video can be added to the new time code track.

[0060] In some optional implementations of this embodiment, after the digital human synthesis task is completed, the task management list can be called out to query the completed tasks, which include the digital human synthesis video and .srt type subtitle files. The digital human synthesis video can be added to the new time code track with one click through the icon. Then, the position and size of each element in the digital human synthesis video can be adjusted in the rendering window of the real-time graphics quick editing packaging tool. Since the digital human synthesis video is just a two-dimensional video piece, after selecting the digital human, the digital human can be scaled to a suitable scale, moved to a suitable perspective position, the digital human mask information can be adjusted, and the digital human can also be matted. For example, in a virtual live studio scene, the digital human can be matted first, and then the digital human can be moved between the virtual large screen and the anchor table. Then the useless part of the edge of the digital human is shielded, and the digital human can be deformed from the whole body to the half body, which is more convenient to complete the spatial design of the virtual lens in the three-dimensional scene.

[0061] The above-mentioned real-time graphic quick-editing packaging tool embedded with the digital human plug-in has the ability to generate AI digital humans; combined with the actual business process, it reduces the production links and tedious and error-prone steps; improves the flexibility of post-production of programs, and has the ability to automatically synthesize subtitles; and provides basic technical support for AI automatic packaging.

[0062] The method for generating digital human synthesized video provided by the above-mentioned embodiments of the present disclosure can improve the flexibility of post-production of programs by setting the side angle; through the .srt subtitle file, it has the ability to automatically synthesize subtitles; in terms of automatic packaging of digital humans, based on the analysis of subtitle content, the packaging animation can be automatically implanted according to the time code position.

[0063] Further references Figure 4 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of a device for generating a digital human synthetic video. Figure 1 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0064] like Figure 4 As shown, the digital human synthesized video generation device 400 of this embodiment includes: an image determination unit 401, a text acquisition unit 402, a scene determination unit 403, an expression determination unit 404 and a digital human synthesis unit 405.

[0065] The image determination unit 401 is configured to determine the virtual image of the digital human according to the configuration information of the digital human.

[0066] The text acquisition unit 402 is configured to acquire the text content of the digital human's spoken script.

[0067] The scene determination unit 403 is configured to determine a matching three-dimensional scene based on the text content.

[0068] The expression determination unit 404 is configured to determine the expression information of the digital human based on the virtual image, text content and three-dimensional scene.

[0069] The digital human synthesis unit 405 is configured to perform a digital human synthesis task based on the expression information to obtain a synthesized video of the digital human.

[0070] In addition, in the technical solution of the present application, an electronic device is also proposed.

[0071] Figure 5 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown.

[0072] like Figure 5 As shown, the electronic device may include a processor 501, a memory 502, a bus 503, and a computer program stored in the memory 502 and executable on the processor 501, wherein the processor 501 and the memory 502 communicate with each other through the bus 503. When the processor 501 executes the computer program, the steps of implementing the above method include, for example: determining the virtual image of the digital human according to the configuration information of the digital human; obtaining the text content of the oral script of the digital human; determining the matching three-dimensional scene based on the text content; determining the expression information of the digital human based on the virtual image, text content and three-dimensional scene; and performing the digital human synthesis task based on the expression information to obtain the synthesized video of the digital human.

[0073] In addition, in one embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented, for example, including: determining a virtual image of the digital human according to the configuration information of the digital human; obtaining the text content of the oral script of the digital human; determining a matching three-dimensional scene based on the text content; determining the expression information of the digital human based on the virtual image, text content and three-dimensional scene; and performing a digital human synthesis task based on the expression information to obtain a synthesized video of the digital human.

[0074] In summary, in the technical solution disclosed in the present invention, the image and oral script of the digital human can be directly configured, and the green background video of the digital human can be combined with the three-dimensional scene, so that the configuration of the digital human and the post-processing of the synthesized video can be integrated into one step, thereby improving the production efficiency of the digital human synthesized video.

[0075] The above description is only a preferred embodiment of the present disclosure and is not intended to limit the present disclosure. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present disclosure shall be included in the scope of protection of the present disclosure.

Claims

1. A method for generating a digital human synthetic video, comprising: Determine the virtual image of the digital human according to the configuration information of the digital human; Obtaining the text content of the digital human's spoken script; Based on the text content, determining a matching three-dimensional scene; Determining expression information of the digital human based on the virtual image, the text content and the three-dimensional scene; Based on the expression information, a digital human synthesis task is performed to obtain a synthesized video of the digital human.

2. The method according to claim 1, wherein: The determining of a matching three-dimensional scene based on the text content includes: Extracting keywords from the text content; Determine matching three-dimensional materials according to the keyword and a preset three-dimensional material resource library; A matching three-dimensional scene is determined according to the three-dimensional material.

3. The method according to claim 2, wherein: The determining of a matching three-dimensional scene based on the text content includes: The three-dimensional scene is determined according to the text content and a preset ratio of the composite video.

4. The method according to claim 1, wherein: The step of determining the expression information of the virtual image according to the virtual image, the text content and the three-dimensional scene includes: Determining an appropriate voice tone according to the virtual image; Determining an adaptive speech rate according to the text content; The action of the virtual image is determined according to the text content and / or the three-dimensional material in the three-dimensional scene.

5. The method according to claim 4, wherein: The step of determining the expression information of the virtual image according to the virtual image, the text content and the three-dimensional scene includes: According to the three-dimensional scene, the sideways angle of the digital human is determined.

6. The method according to claim 4, wherein: The method further comprises: Generate a subtitle file according to the text content and the synthesized video; Subtitles corresponding to the text content are displayed in the composite video.

7. The method according to claim 6, wherein: The step of displaying subtitles corresponding to the text content in the composite video includes: The size and arrangement of the fonts in the subtitles are determined according to the width of the composite video.

8. The method according to claim 1, wherein: The step of performing a digital human synthesis task based on the expression information to obtain a synthesized video of the digital human includes: Determining a green background video of the digital human according to the expression information; Determine a key frame image according to the green background video, the three-dimensional scene and the camera movement information; Rendering is performed according to the key frame images to obtain the composite video.

9. The method according to claim 1, wherein: The method further comprises: The synthesis task of the digital human is executed in the background, and the execution progress of the synthesis task is displayed.

10. The method according to claim 7, wherein: The method further comprises: In response to receiving adjustment information for at least one item in the composite video, real-time rendering is performed according to the adjustment information.

11. A digital human synthesis device, comprising: An image determination unit is configured to determine a virtual image of the digital human according to the configuration information of the digital human; A text acquisition unit, configured to acquire the text content of the oral script of the digital human; A scene determination unit, configured to determine a matching three-dimensional scene based on the text content; an expression determination unit, configured to determine expression information of the digital human based on the virtual image, the text content and the three-dimensional scene; The digital human synthesis unit is configured to perform a digital human synthesis task based on the expression information to obtain a synthesized video of the digital human.

Images