A method and system for dynamic synthetic video based on digital humans

Abstract

The present application relates to the technical field of dynamic video synthesis, in particular to a method and system for synthesizing dynamic video based on digital people, which comprises the following specific steps: S1, inputting files containing background, ppt, digital people video and digital people idle animation video, and judging whether the background file is a video file or a picture file; S2, respectively converting the background and the ppt video; S3, synthesizing the ppt video and the background video to obtain a basic video; S4, synthesizing the digital people video and the basic video to obtain a final video; and S5, outputting the final video as an output file. The present application can automatically perform scene judgment according to the time length of each component (the time length of the background video, the time length of the ppt video and the time length of the digital people voice broadcast video) to dynamically synthesize the video, and simultaneously supports the scene of the ppt chapter being a video, thereby greatly improving the experience of the user.

Description

Technical Field

[0001] This invention relates to the field of dynamic video synthesis technology, and in particular to a method and system for dynamic video synthesis based on digital humans. Background Technology

[0002] With the continuous development of science and technology, the application scenarios for virtual digital humans are increasing, including the scenario of converting PowerPoint presentations into videos and then having a digital human read the content. Currently available conventional methods can achieve simple PowerPoint-to-video conversion, but they have the following problems:

[0003] ① PowerPoint only supports images, not videos;

[0004] ② If a video is selected as the background, it cannot dynamically expand according to the duration of the PPT chapters that are videos or digital human voice narrations. This will cause the background video to suddenly freeze before the PPT chapters or digital human videos have finished playing, greatly reducing the user experience.

[0005] ③ If the duration of the PPT video is greater than the duration of the digital human's voice narration video, then the digital human will remain still after it finishes narrating, but the PPT video has not yet finished playing, which greatly reduces the user experience.

[0006] To address the above issues, a method and system for dynamically synthesizing videos from PPT slides based on digital humans becomes particularly important. Summary of the Invention

[0007] The purpose of this invention is to address the problems existing in the background technology by proposing a method and system for dynamic synthetic video based on digital humans.

[0008] The technical solution of this invention, a method for dynamic synthetic video based on digital humans, includes the following specific steps:

[0009] S1. Input a file containing a background, a PPT, a digital human video, and a digital human idle animation video, and determine whether the background file is a video file or an image file.

[0010] S2. Convert the background and PPT video separately;

[0011] S3. Combine the PPT video and the background video to obtain the base video;

[0012] S4. The final video is obtained by combining the digital human video with the basic video.

[0013] S5. Output the final video as an output file.

[0014] Preferably, in S1, it is determined whether the background component is a video or an image; if it is a video, it is determined whether the duration of the background video is ≥1 minute; if so, the file is output; otherwise, the background video is created.

[0015] If it is an image file, the image file will be made into a 1-minute video.

[0016] Preferably, if the background component is an image file, ffmpeg is used to convert the background image into a 1-minute background video, and the converted file is then output.

[0017] Preferably, if the background video duration is less than 1 minute, the duration of the background video is read and recorded as bgTime; the time difference obtained by subtracting bgTime from 60 is recorded as tmp; Math.ceil((double)tmp / bgTime) is used to obtain the number of times the background video needs to be looped, which is recorded as count; at this time, ffmpeg is used to create the background video: the background video is looped count times to obtain the final background video.

[0018] Preferably, the PPT component is obtained from the input file, and it is determined whether the PPT component is a video or an image: if the PPT component is a video, the file is output; if it is an image, the PPT image is converted into a 5-second video.

[0019] Preferably, ffmpeg is used to convert PPT images into a 5-second PPT video, and then the output file is generated after conversion.

[0020] Preferably, in S3, the PPT video and the background video are combined into a video using ffmpeg according to the required X and Y parameters, and the combined video is recorded as the base video.

[0021] Preferably, the base video and the digital human video are dynamically composited;

[0022] Determine if the length of the digital human video is longer than the length of the base video. If so, the base video needs to be remade. Otherwise, determine if the length of the digital human video is longer than the length of the PowerPoint video. If it is longer than the PowerPoint video, extract the background color of the digital human video using ffmpeg to obtain video A. Then, combine video A with the base video using ffmpeg according to the X and Y parameters to obtain video B. After video B is generated, trim video B according to the length of the digital human video using ffmpeg to obtain video C. At this point, video C is the final output file.

[0023] If the duration is not greater than the PPT video length, then supplement the time gap with idle animation videos of the digital human. Then, stitch the digital human video and the idle animation videos together to create a new digital human voice broadcast video, denoted as Video A. After the video is stitched, use ffmpeg to perform background color removal on Video A to obtain Video B. After the background removal is completed, use ffmpeg to synthesize Video B and the base video in S10 according to the required X and Y parameters. The synthesized video is denoted as Video C. After the video is successfully synthesized, use ffmpeg to trim Video C according to the duration of the PPT video to obtain the final output video file Video D.

[0024] The preferred method for remaking the basic video is as follows:

[0025] Determine if the background component is a video; if it is, then:

[0026] The duration of the digital human video read is denoted as szrTime;

[0027] The duration of reading the background video is denoted as bgTime.

[0028] Calculate the time difference between the two, i.e., szrTime-bgTime, denoted as loopTime;

[0029] Use Math.ceil((double)loopTime / bgTime) to get the number of times the background video needs to loop, and record it as count;

[0030] To create a new background video using ffmpeg: loop through the background video count times to get the video tmp, then trim the video tmp according to szrTime to get the final new background video A;

[0031] If the background component is an image:

[0032] Use ffmpeg to convert the background image into a background video with the same duration as the digital human, denoted as video A. After conversion, combine the PPT video with background video A to obtain a new base video B.

[0033] Video C is obtained by background color extraction from the digital human video;

[0034] Video C and the base video B are combined using ffmpeg according to the required X and Y coordinate parameters to obtain the final video D, which is the final output file.

[0035] A system for dynamically synthesized videos based on digital humans, which operates according to the above method, outputs the final video file.

[0036] Compared with the prior art, the present invention has the following beneficial technical effects:

[0037] 1. If the background component is a video, it can be dynamically expanded according to the duration of the video or the digital human voice narration video in the PPT chapter;

[0038] 2. If the PPT is a video and its duration is longer than the duration of the digital human's voice narration video, idle animations will be automatically filled to make up for the time difference.

[0039] 3. Supports scenarios where PPT chapters are videos;

[0040] 4. Users can select different background videos or images according to their needs, and the system will dynamically expand and synthesize them. Attached Figure Description

[0041] Figure 1 This is a diagram showing the final practical effect of the video synthesis method of the present invention;

[0042] Figure 2 This is a flowchart illustrating the overall module implementation of the present invention;

[0043] Figure 3 This is a flowchart of steps three to six in an embodiment of the present invention;

[0044] Figure 4 This is a flowchart of steps seven to eight in an embodiment of the present invention;

[0045] Figure 5 Flowcharts of steps nine to thirteen in this embodiment of the invention;

[0046] Figure 6 This is a flowchart of step fourteen in an embodiment of the present invention;

[0047] Figure 7 This is a flowchart of steps 15 to 20 in an embodiment of the present invention. Detailed Implementation

[0048] Example 1

[0049] This invention proposes a method for dynamic synthetic video based on digital humans, comprising:

[0050] Step 1: The final result of this solution is shown in the image below. Figure 1 As shown;

[0051] The background is the background of the final composite video, and it can be an image or a video.

[0052] The ppt position represents the content of the current ppt page, which can be either an image or a video;

[0053] The digital human location display shows the digital human's voice and video that needs to be played on the current PPT page.

[0054] Step Two: The overall module implementation flowchart of this solution is as follows Figure 2 As shown:

[0055] The following steps are described using a scenario from a single slide in a PowerPoint presentation.

[0056] Step 3: Retrieve the four components contained in the input file (background, PPT, digital human video, and digital human idle animation video); determine whether the background component is a video: if it is a video, proceed to step 4; if it is an image, proceed to step 5.

[0057] Step 4: If the background component is a video, proceed to this step and determine if the duration of the background video is ≥1 minute: If the background duration is ≥1 minute, proceed to Step 7; otherwise, proceed to Step 6.

[0058] Step 5: Proceed to this step if the background component is an image. Use ffmpeg to convert the background image into a 1-minute background video. After the conversion is complete, proceed to Step 7.

[0059] Step Six: This step is entered when the background component is a video and the background video duration is <1 minute. The duration of the background video is read and denoted as bgTime (in seconds). The time difference between 60 and bgTime is denoted as tmp. Math.ceil((double)tmp / bgTime) is used to obtain the number of times the background video needs to be looped, denoted as count. Then, ffmpeg is used to create the background video: the background video is looped count times to obtain the final background video. Proceed to Step Seven. (Flowcharts from Steps Three to Six are shown.) Figure 3 As shown;

[0060] Step 7: Reaching this step indicates that a background video meeting the requirements has been obtained. At this point, the PPT component is retrieved from the input file, and it is determined whether the PPT component is a video: if it is a video, proceed to Step 9; otherwise, proceed to Step 8.

[0061] Step 8: Proceed to this step if the PPT components are images. Use ffmpeg to convert the PPT images into a 5-second PPT video. After conversion, proceed to Step 9. The flowchart for Steps 7 to 8 is as follows: Figure 4 As shown:

[0062] Step Nine: Reaching this step indicates that you have obtained a PPT video that meets the requirements. Now, use ffmpeg to combine the PPT video and the background video obtained in Step Seven into a composite video according to the required X and Y parameters. The composite video is recorded as the base video.

[0063] Step 10: Reaching this step indicates that the basic video has been generated. Now you are ready to perform dynamic video compositing with the digital human video (digital human idle animation video).

[0064] Step 11: Determine if the digital human video length is greater than the basic video length. If it is, proceed to Step 15; otherwise, proceed to Step 12.

[0065] Step 12: Determine if the length of the digital human video is greater than the length of the PPT video: if it is, proceed to Step 14; otherwise, proceed to Step 13.

[0066] Step Thirteen: This step indicates that the digital human video duration is less than the base video duration and less than the PPT video duration. At this point, add the idle animation video of the digital human to fill in the time difference. Then, stitch the digital human video and the idle animation video together to create a new digital human voice broadcast video, denoted as Video A. After video stitching, use ffmpeg to perform background removal on Video A to obtain Video B. After background removal, combine Video B and the base video from Step Ten using ffmpeg according to the required X and Y parameters. The resulting composite video is denoted as Video C. After successful composite video creation, trim Video C according to the PPT video duration using ffmpeg to obtain the final output video file, Video D. The flowchart is as follows: Figure 5 As shown:

[0067] Step Fourteen: This step indicates that the digital human video duration is less than the base video duration and greater than the PPT video duration. At this point, the digital human video is used to perform background color extraction using ffmpeg to obtain Video A. Then, Video A is combined with the base video using ffmpeg based on the X and Y parameters to obtain Video B. After Video B is generated, it is trimmed using ffmpeg according to the digital human video duration to obtain Video C. Video C is the final output file. The flowchart is as follows. Figure 6 As shown:

[0068] Step 15: If you enter this step, it means that the duration of the digital human video is greater than the duration of the basic video. At this time, you need to recreate the basic video. We have already obtained the PPT video we need from Step 7 or Step 8. Now, determine whether the background component is a video. If it is a video, proceed to Step 16; otherwise, proceed to Step 17.

[0069] Step Sixteen: This step indicates that the background component is a video. You now need to create a background video of the same length as the digital human's video. The method is as follows:

[0070] ① The duration of reading the digital human video is recorded as szrTime (unit: seconds);

[0071] ② The duration of reading the background video is recorded as bgTime (in seconds);

[0072] ③ Calculate the time difference between the two, i.e., szrTime-bgTime, denoted as loopTime;

[0073] ④ Use Math.ceil((double)loopTime / bgTime) to get the number of times the background video needs to loop, and record it as count;

[0074] ⑤ Use ffmpeg to create a new background video: loop the background video count times to get the video tmp, and then trim the video tmp according to szrTime to get the final new background video A;

[0075] After the new background video A is completed, proceed to step eighteen;

[0076] Step 17: This step indicates that the background component is an image. At this point, use ffmpeg to convert the background image into a background video with the same duration as the digital human, denoted as Video A. After the conversion is complete, proceed to Step 18.

[0077] Step 18: Entering this step indicates that the new background video A has been completed. At this point, combine the PPT video mentioned in Step 15 with the background video A to obtain the new base video B, and then proceed to Step 19.

[0078] Step 19: Remove the background color from the digital human video to obtain video C, then proceed to step 20;

[0079] Step 20: Combine video C and base video B using ffmpeg according to the required X and Y coordinate parameters to obtain the final video D. Video D is the final output file. The flowchart is as follows: Figure 7 As shown;

[0080] In this embodiment, the video output using the above method can dynamically perform scene judgment and dynamic video synthesis based on the duration of each component (background video duration, PPT video duration, and digital human voice broadcast video duration). It also supports scenes where PPT chapters are videos, greatly improving the user experience.

[0081] The embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the present invention is not limited thereto. Various changes can be made within the scope of knowledge possessed by those skilled in the art without departing from the spirit of the present invention.

Claims

1. A method for dynamically synthesizing videos based on digital humans, characterized in that, The method takes a single slide from a PowerPoint presentation as the processing object and includes the following specific steps: S1. Input a file containing a background, a PPT, a digital human video, and a digital human idle animation video, and determine whether the background file is a video file or an image file. S2. Convert the background and PPT to generate background video and PPT video respectively; S3. Combine the PPT video and the background video to obtain the base video; S4. The final video is obtained by combining the digital human video with the basic video. The specific steps include: Determine if the length of the digital human video is longer than the length of the base video. If so, the base video needs to be remade. Otherwise, determine if the length of the digital human video is longer than the length of the PowerPoint video. If it is longer than the PowerPoint video, extract the background color of the digital human video using ffmpeg to obtain video A. Then, combine video A with the base video using ffmpeg according to the X and Y parameters to obtain video B. After video B is generated, trim video B according to the length of the digital human video using ffmpeg to obtain video C. At this point, video C is the final output file. If the duration is not greater than the PPT video length, then supplement the time gap with idle animation videos of the digital human. Then, stitch the digital human video and the idle animation videos together to create a new digital human voice broadcast video, denoted as Video A. After the video is stitched, use ffmpeg to perform background color removal on Video A to obtain Video B. After the background removal is completed, use ffmpeg to synthesize the video with the base video according to the required X and Y parameters. The synthesized video is denoted as Video C. After the synthesized video is successfully completed, use ffmpeg to trim Video C according to the duration of the PPT video to obtain the final output video file Video D. S5. Output the final video as an output file.

2. The method for dynamic synthetic video based on digital humans according to claim 1, characterized in that, In S1, determine whether the background component is a video or an image; if it is a video, determine whether the background video duration is ≥1 minute; if yes, output the file; otherwise, create the background video. If it is an image file, the image file will be made into a 1-minute video.

3. The method for dynamic video synthesis based on digital humans according to claim 2, characterized in that, If the background component is an image file, ffmpeg is used to convert the background image into a 1-minute background video, and the output file is generated after the conversion.

4. The method for dynamic video synthesis based on digital humans according to claim 2, characterized in that, If the duration of the background video is less than 1 minute, the duration of reading the background video is recorded as bgTime; The time difference between 60 and bgTime is denoted as tmp; Math.ceil((double) tmp / bgTime) is used to get the number of times the background video needs to be looped, denoted as count; then ffmpeg is used to create the background video: the background video is looped count times to get the final background video.

5. The method for dynamic synthetic video based on digital humans according to claim 1, characterized in that, The PPT component is retrieved from the input file. The PPT component is then determined to be either a video or an image. If the PPT component is a video, the output file is generated. If it is an image, the PPT image is converted into a 5-second video.

6. The method for dynamic synthetic video based on digital humans according to claim 5, characterized in that, Use ffmpeg to convert PPT images into a 5-second PPT video, and then output the converted file.

7. The method for dynamic synthetic video based on digital humans according to claim 1, characterized in that, In S3, the PPT video and background video are combined into a video using ffmpeg according to the required X and Y parameters. The combined video is called the base video.

8. The method for dynamic synthetic video based on digital humans according to claim 1, characterized in that, The method for remaking the basic video is as follows: Determine if the background component is a video; if it is, then: The duration of the digital human video read is denoted as szrTime; The duration of reading the background video is denoted as bgTime. Calculate the time difference between the two, i.e., szrTime - bgTime, denoted as loopTime; Use Math.ceil((double) loopTime / bgTime) to get the number of times the background video needs to loop, and denot it as count; To create a new background video using ffmpeg: loop through the background video count times to get the video tmp, then trim the video tmp according to szrTime to get the final new background video A; If the background component is an image: Use ffmpeg to convert the background image into a background video with the same duration as the digital human, denoted as video A. After conversion, combine the PPT video with background video A to obtain a new base video B. Video C is obtained by background color extraction from the digital human video; Video C and the base video B are combined using ffmpeg according to the required X and Y coordinate parameters to obtain the final video D, which is the final output file.

9. A system for dynamic synthetic video based on digital humans, characterized in that, The method described in any one of claims 1-8 is executed to output a final video file.

Images