A digital human automatic explanation static PPT video production method

Abstract

The application provides a digital person automatic explanation of static PPT video production method and device, computer equipment and computer storage medium, and relates to the technical field of video processing. Including: extracting first target text information in each sub-image from a sequence image set of static PPT; converting the first target text information in each sub-image into speech, and synthesizing the obtained initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file to construct a corresponding relationship between the audio time of each initial audio file and the display time of each sub-image; adding a PPT transition animation with a preset pause duration between each sub-image to obtain an initial PPT video for the static PPT; and based on the target audio file, a virtual digital person is controlled to comment on the first target text information in the initial PPT video to obtain a target PPT video. The method can improve the smoothness of video playback.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video processing, in particular to a video production method for digital person automatic explanation of static PPT, a video production device for digital person automatic explanation of static PPT, a computer device and a computer storage medium. BACKGROUND

[0002] With the continuous development of the Internet and virtual technology, the application scenarios of virtual digital person are more and more, which include a scenario of converting PPT into video and broadcasting PPT content through virtual digital person. However, each picture in the PPT is a static picture, so a transition animation needs to be added in the process of converting the PPT into a video, so as to realize a smooth transition effect, otherwise, the video will be stuck in transition, thereby affecting the video production fluency of the digital person automatic explanation of the PPT.

[0003] In the related technical solution, the PPT image is usually taken as a background, and a small section of video is synthesized by using audio to drive the digital person to tell a small section of PPT animation to make each PPT into a small section of video, and finally a complete explanation video is synthesized. However, the above process still has the problem of video sticking in PPT transition, thereby affecting the fluency of video playing. SUMMARY

[0004] The embodiments of the present application provide a video production method for digital person automatic explanation of static PPT, a video production method for digital person automatic explanation of static PPT, a computer device and a computer storage medium, thereby at least to some extent overcoming the technical problem that the video based on static PPT has the problem of video sticking in PPT transition when playing, thereby affecting the fluency of video playing due to the limitations and defects of the related technology.

[0005] The first aspect of the embodiments of the present application provides a video production method for digital person automatic explanation of static PPT, the method comprising: extracting first target text information in each sub-image from a sequence image set of static PPT; converting the first target text information in each sub-image into speech to obtain an initial audio file corresponding to each sub-image, and synthesizing the initial audio file corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of the corresponding each sub-image in the static PPT is constructed; adding a PPT transition animation with a preset pause duration between each sub-image to obtain an initial PPT video for the static PPT; adding a virtual digital person on the initial PPT video, and controlling the virtual digital person to comment on the first target text information in the initial PPT video based on the target audio file to obtain a target PPT video.

[0006] In a second aspect, the application provides a device for automatically explaining a static PPT by a digital human, comprising: a text extraction module configured to extract first target text information in each sub-image from a sequence of images of the static PPT; an audio synthesis module configured to convert the first target text information in each sub-image into speech to obtain an initial audio file corresponding to each sub-image, and to synthesize the initial audio file corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of the corresponding sub-image in the static PPT is constructed; an animation adding module configured to add a PPT transition animation with a preset pause duration between each sub-image to obtain an initial PPT video for the static PPT; and a control module configured to add a virtual digital human to the initial PPT video, and to control the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file to obtain a target PPT video.

[0007] In a third aspect, the application provides a computer device, comprising a memory and a processor, the memory storing a computer program, and the processor implementing the steps of any of the above methods for automatically explaining a static PPT by a digital human when executing the computer program.

[0008] In a fourth aspect, the application provides a computer readable storage medium, having a computer program stored thereon, the computer program being executed by a processor to implement the steps of any of the above methods for automatically explaining a static PPT by a digital human.

[0009] In a fifth aspect, the application provides a computer program product, comprising a computer program, the computer program being executed by a processor to implement the steps of any of the above methods for automatically explaining a static PPT by a digital human.

[0010] The technical solution of the application has the following advantages:

[0011] The video production method for automatic explanation of digital people to static PPT extracts first target text information in each sub-image from a sequence image set of the static PPT; converts the first target text information in each sub-image into speech to obtain initial audio files corresponding to each sub-image, and synthesizes the initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between audio times of each initial audio file and display times of corresponding sub-images in the static PPT is constructed; a PPT transition animation with a preset pause duration is added between each sub-image to obtain an initial PPT video for the static PPT; a virtual digital person is added to the initial PPT video, and the virtual digital person is controlled to comment on the first target text information in the initial PPT video based on the target audio file to obtain a target PPT video. On the one hand, the method automatically converts the text information corresponding to each sub-image obtained from the static PPT into audio, and drives the virtual digital person to automatically comment, so that the text information corresponding to each sub-image corresponds to the content of the virtual digital person's commentary, thereby improving the accuracy of the static PPT commentary. On the other hand, the corresponding relationship between the audio times of the initial audio files and the display times of the corresponding sub-images in the static PPT is constructed, and the virtual digital person is controlled to comment on the first target text information in the initial PPT video based on the target audio file, so that the background displayed PPT animation and the video animation of the virtual digital person's commentary are kept synchronous to be combined together through the time relationship, thereby avoiding the stuttering problem caused by the harsh PPT transition, and ensuring the smoothness of the video playback. BRIEF DESCRIPTION OF DRAWINGS

[0012] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiments of the application and serve to explain the principles of the application, and do not limit the application. In the drawings:

[0013] Figure 1 A flowchart of one of the video production methods for automatic explanation of digital people to static PPT provided by an embodiment of the application;

[0014] Figure 2 A schematic diagram of the generated first target text information provided by one of the embodiments of the application;

[0015] Figure 3 A schematic diagram of the synthesized target audio file provided by one of the embodiments of the application;

[0016] Figure 4 A PPT switching setting page schematic diagram provided by one of the embodiments of the application;

[0017] Figure 5 A candidate digital human figure diagram provided for an embodiment of the present application;

[0018] Figure 6 A video file diagram in which a virtual digital human driven by audio tells a static PPT provided for an embodiment of the present application;

[0019] Figure 7 A target PPT video diagram provided for an embodiment of the present application;

[0020] Figure 8 A video production device structure diagram in which a digital human automatically explains a static PPT provided for an embodiment of the present application;

[0021] Figure 9 A computer device structure diagram provided for an embodiment of the present application. DETAILED DESCRIPTION

[0022] Example implementations will now be described more fully with reference to the accompanying drawings. Example implementations may, however, be implemented in many different forms and should not be construed as limited to the implementations set forth herein; rather, these implementations are provided so that this disclosure will be thorough and complete, and will fully convey the inventive aspects to those skilled in the art. The described features, structures, or characteristics can be combined in one or more implementations. In the following description, numerous specific details are provided to give a thorough understanding of implementations of the disclosure. One skilled in the relevant art will recognize, however, that the implementations of the disclosure can be practiced without one or more of the specific details, or with other methods, components, materials, and so forth. In other instances, well-known structures have not been described in detail to avoid obscuring aspects of the disclosure.

[0023] Furthermore, the drawings are not necessarily drawn to scale. Like reference numbers in the drawings indicate like or similar elements so that the numerous illustrative implementations will be understood as such even without being referred to in connection with discussing a specific implementation. Some of the blocks in the flowchart illustrations can be implemented as software modules or portions of software code, for example. Such software modules can be implemented in hardware, firmware, and / or software.

[0024] The flowchart illustrations shown in the drawings are examples only and are not necessarily required to implement the methods illustrated by the flowcharts. For example, one or more of the steps can be combined or partially combined, and some steps can be omitted, so that the actual order of execution can vary from the order shown.

[0025] In the related technical background, with the continuous development of the Internet and virtual technology, the application scenarios of virtual digital people are more and more, thereby generating artificial intelligence (AI) virtual image products in various fields. For example, a scenario of converting PPT into a video and broadcasting PPT content through a virtual digital person. In existing AI virtual image products, the general operation process is anchor video acquisition, data processing, model training, and image output; wherein, in the output stage of production, based on the trained anchor image, according to the input script, voice, and optionally anchor emotion, background picture, video, posture, and the like, video generation output is performed. However, in the current AI digital person video generation system, data flows in one direction, and only pictures and videos can be input as backgrounds and synthesized with AI digital people. In the above application scenario, the characteristics of static PPT are similar to still pictures, and in the process of converting PPT into a video, only transition animation can be added to make the transition smooth, otherwise the transition will be stuck, thereby affecting the smoothness of the virtual digital person automatically explaining the PPT video production.

[0026] In the existing technical method, the PPT file is converted into an image file, and the virtual digital person is driven by audio to tell the content of each PPT to form a small animation, and finally the above plurality of small videos are synthesized into a complete video. However, the video produced by the above method is still the same as the direct conversion of static pictures into a video, which will cause the technical problem of transition lag, thereby affecting the smoothness of the virtual digital person automatically explaining the PPT content video.

[0027] The embodiment of the present application considers the above technical problems and proposes a digital person automatic explanation of static PPT video production method. The method can be applied to any application scenario that needs to use a virtual digital person for video production. The method automatically converts the text information corresponding to each sub-image obtained from the static PPT into audio, and drives the virtual digital person to automatically comment through the audio, so that the text information corresponding to each sub-image corresponds to the content of the virtual digital person's commentary, thereby improving the accuracy of the static PPT explanation. On the other hand, the corresponding relationship between the audio time of the initial audio file and the display time of the corresponding sub-image in the static PPT is constructed, and the virtual digital person is controlled based on the target audio file to explain the first target text information in the initial PPT video, thereby ensuring that the PPT animation displayed in the background and the video animation of the virtual digital person's explanation are kept in synchronization to be combined together through the time relationship, thereby avoiding the lag problem caused by the harsh PPT transition, thereby ensuring the smoothness of the video playback.

[0028] The following method can be executed by a terminal device or a server, or by a terminal device and a server together, which is specifically illustrated here as an example of a terminal device or a server as an execution subject.Figure 1 A flowchart of a video production method for automatically explaining a static PPT by a digital person according to an embodiment of the present application is shown in FIG. 1, which includes the following steps 101 to 104. Figure 1

[0029] Step 101: Extract first target text information in each sub-image from a sequence image set corresponding to the static PPT.

[0030] Step 102: Convert the first target text information in each sub-image into speech to obtain an initial audio file corresponding to each sub-image, and synthesize the initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT.

[0031] In the target audio file, a preset pause duration is added between each initial audio file, and a corresponding relationship between audio time of each initial audio file and display time of the corresponding sub-image in the static PPT is constructed.

[0032] Step 103: Add a PPT transition animation with a preset pause duration between each sub-image to obtain an initial PPT video for the static PPT.

[0033] Step 104: Add a virtual digital person to the initial PPT video, and control the virtual digital person to comment on the first target text information in the initial PPT video based on the target audio file to obtain a target PPT video.

[0034] Through the above Figure 1 The embodiment shown in FIG. 1 automatically converts the text information corresponding to each sub-image obtained from the static PPT into audio, and drives a virtual digital person to automatically comment on the text information, so that the text information corresponding to each sub-image corresponds to the content of the virtual digital person's commentary, thereby improving the accuracy of the static PPT commentary. On the other hand, the corresponding relationship between the audio time of the initial audio file and the display time of the corresponding sub-image in the static PPT is constructed, and the virtual digital person is controlled to comment on the first target text information in the initial PPT video based on the target audio file, so that the PPT animation in the background display and the video animation of the virtual digital person's commentary are kept synchronized to be combined together through the time relationship, thereby avoiding the stuttering problem caused by the harsh PPT transition, and ensuring the smoothness of the video playback.

[0035] The following will combine specific embodiments to introduce each step shown in the above Figure 1 in detail.

[0036] In step 101, the first target text information in each sub-image is extracted from a sequence image set corresponding to the static PPT.

[0037] ​The static PPT includes multiple static PPT pages. For example, the first target text information in each sub-image can be extracted from a corresponding sequence image set of the multiple static PPT pages.

[0038] In the step 101, the static PPT includes multiple static PPT pages, and is not an image file. Therefore, the multiple static PPT pages need to be converted into image files to obtain multiple sub-images corresponding to the multiple static PPT pages, and obtain the sequence image set. The following will be described in detail with reference to specific embodiments.

[0039] In an optional embodiment of the present disclosure, a static PPT is obtained, the static PPT is stored in an image format to obtain a sequence image set corresponding to the static PPT, and the first target text information in each sub-image is extracted from the sequence image set corresponding to the static PPT.

[0040] For example, the PPT file can be directly saved as an image file to realize the conversion process from the PPT file to the image file.

[0041] In an optional embodiment of the present disclosure, a screenshot operation is performed on each PPT page in the static PPT to obtain a sequence image set corresponding to the static PPT, and the first target text information in each sub-image is extracted from the sequence image set corresponding to the static PPT.

[0042] For example, the multiple PPT pages in the PPT file can be respectively subjected to a screenshot operation to obtain corresponding images, and then a sequence image set corresponding to the static PPT is obtained.

[0043] In an optional embodiment of the present disclosure, the static PPT is processed based on a preset conversion model or a class file to obtain a sequence image set corresponding to the static PPT, and the first target text information in each sub-image is extracted from the sequence image set corresponding to the static PPT.

[0044] For example, the image conversion operation of the static PPT can be realized based on a preset conversion model trained in advance, or the image conversion operation of the static PPT can be realized using a corresponding class file. For example, a related class file in the Apache POI library can be imported using JAVA to process the static PPT to obtain the sequence image.

[0045] Through the above embodiments, the PPT page is converted into an image file, which facilitates the extraction of text information in the image in the subsequent process, so as to accurately and quickly obtain the first target text information in each sub-image, and improve the accuracy of the virtual digital person in explaining the first target text information in each sub-image.

[0046] After obtaining the sequence image set corresponding to the presentation PPT based on the above embodiments, the first target text information in each sub-image can be extracted from the sequence image set corresponding to the static PPT.

[0047] In one optional embodiment of this disclosure, initial text information is extracted from each sub-image from the sequence of images corresponding to the static PPT; the initial text information is expanded based on the image-text multimodal understanding model to obtain the first target text information for each sub-image.

[0048] Among them, the image-text multimodal understanding model is a large model used to understand and expand the image content. For example, the image-text multimodal understanding model can be the CogVLM multimodal large model, the Tongyi Qianwen model, the Wenxin Yiyan model, etc., and the embodiments of this application do not impose any limitations on it.

[0049] For example, the initial text information is derived from the text information contained in each sub-image, while the first target text information is obtained by expanding the content of the initial text information based on an image-text multimodal understanding model. When explaining a PowerPoint presentation, simply explaining the initial text information in the static PowerPoint slide is too rigid and inflexible. To better meet actual presentation needs, the extracted initial text information can be expanded.

[0050] For ease of understanding, the following will combine Figure 2 The above embodiments are illustrated by way of example based on the first target text information in each sub-image.

[0051] Figure 2 This is a schematic diagram illustrating one type of generated first target text information provided in an embodiment of this application, with reference to... Figure 2 The graphics and text displayed on the upper part of the PPT slide only provide the initial text information such as "Table of Contents" and "1. Work Overview; 2. Completion Status; 3. Results Presentation; 4. Shortcomings; 5. Work Plan." This initial text information only contains prompts. By expanding the content of these prompts using the CogVLM multimodal large model, we can obtain... Figure 2 The first target text information in the lower half is used so that the virtual digital human can explain the static PPT based on the first target text information.

[0052] Depend on Figure 2 As can be seen from the first target text information shown in the lower half, the expanded first target text information is more natural and conforms to the language habits of the actual narrator in the real scene, thus improving the authenticity of the video.

[0053] In step 102, the first target text information in each sub-image is converted into speech to obtain an initial audio file corresponding to each sub-image, and the initial audio files corresponding to each sub-image are synthesized to obtain a target audio file for the static PPT.

[0054] In the target audio file, a preset pause duration is added between each initial audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of the corresponding sub-image in the static PPT is constructed.

[0055] In the target audio file, a preset pause duration is added between each initial audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of the corresponding sub-image in the static PPT is constructed. Figure 2 As shown in the static PPT, the sub-image is continuously displayed for the audio time until the audio time for the sub-image is over. The audio time of the initial audio file includes at least the start time, end time, and playback duration of the initial audio file.

[0056] Through the embodiment, the corresponding relationship between the audio time of each initial audio file and the display time of the corresponding sub-image in the static PPT is constructed, which can avoid the situation that the audio for a sub-image is not played completely before switching to the next sub-image, or the audio for a sub-image has been played completely but still does not switch to the next sub-image, thereby avoiding the transition lag problem in the video playback process.

[0057] For example, the first target text information in each sub-image obtained in step 101 can be converted into speech to obtain an initial audio file corresponding to each sub-image. For example, a speech-to-text conversion model (TTS) method can be used to convert the first target text information in each sub-image into audio; or the Whisper Speech method can be used to convert from text to speech to form audio.

[0058] In the process of converting the first target text information in each sub-image into speech, the first target text information in each sub-image can be converted into speech based on a preconfigured tone to obtain an initial audio file corresponding to each sub-image. Generally, the preconfigured tone is consistent with the selected target digital human image. For example, if the selected target digital human image is a man, the corresponding configured tone is male voice; or if the selected target digital human image is a middle-aged man, the corresponding configured tone is a low male voice.

[0059] In step 102, the initial audio files corresponding to each sub-image are synthesized to obtain a target audio file for the static PPT; and a preset pause duration is added between each initial audio file in the target audio file.

[0060] For example, after obtaining the initial audio files corresponding to the sub-images based on step 102, the initial audio files corresponding to the sub-images can be synthesized to obtain the target audio file for the static PPT.

[0061] The process of synthesizing the target audio file will be exemplarily described below in combination with specific embodiments.

[0062] Embodiment 1: In an optional embodiment of the present disclosure, a preset pause duration is added to each initial audio file corresponding to the sub-images; the initial audio files corresponding to the sub-images and the preset pause duration are synthesized to obtain the target audio file for the static PPT.

[0063] For example, after obtaining the initial audio files corresponding to the sub-images, a preset pause duration (for example, 1 second) can be added to the end of each initial audio file, so that the initial audio files corresponding to the sub-images and the preset pause duration can be synthesized to obtain the target audio file for the static PPT.

[0064] For example, assuming that the static PPT contains 3 PPTs, and the 3 initial audio files corresponding to the 3 PPTs are input1.mp3, input2.mp3 and input3.mp3 based on step 102, the implementation logic of the integrated target audio file is as follows:

[0065] ffmpeg-itsoffset 1-iinput1.mp3-itsoffset 1-iinput2.mp3-itsoffset 1-iinput3.mp3-filter_complex"amix=inputs=3:duration=first"output.mp3.

[0066] Among them:

[0067] -itsoffset 1: indicates that input1.mp3 is played with a delay of 1 second.

[0068] -filter_complex"amix=inputs=3:duration=first": indicates that the three input audio files are mixed, and the duration of the shortest file (i.e. duration=first) is used to determine the length of the final output.

[0069] output.mp3: indicates the output target audio file.

[0070] It needs to be explained that after running the command, the output. mp3 file will contain the audio mixing result of the preset pause length, provided that the paths of input1. mp3, input2. mp3 and input3. mp3 are correct.

[0071] As can be seen from the above embodiment, 1-second delay (i.e. 1-second pause) is added after the 3 initial audio files corresponding to the 3 PPTs, so as to mix the three input audio files and obtain the output target audio file.

[0072] In addition to the above embodiment, in another optional embodiment of the present disclosure, the first target text information in each sub-image is integrated into the same text to obtain integrated second target text information; a preset pause length is added after the end text mark of the first target text information of each sub-image in the second target text information; the second target text information and the preset pause length are converted into speech to obtain the target audio file for the static PPT.

[0073] For example, the first target text information in all sub-images in the PPT text can be integrated into the same text, and a pause mark can be added at the end of the text content of each PPT. Finally, all the text and pauses are converted into audio to obtain the target audio file.

[0074] The following will be described in conjunction with Figure 3 for exemplary illustration. Figure 3 One of the embodiments of the present application provides a schematic diagram for synthesizing a target audio file, as shown in Figure 3 The contents of two PPTs are shown. The target text information of the first PPT is “Hello everyone. Today I will introduce the work results in the past period of time and the future work plan.”; the target text information of the second PPT is “First, let's take a look at our directory. The first part is the work overview, the second part is the completion, the third part is the achievement display, and the last is the work plan.”; then a 1-second pause mark is added at the end of the target text information of the first PPT. Finally, all the text and pauses are converted into audio to obtain the target audio file.

[0075] Through the above embodiment, the corresponding relationship between the audio time of each initial audio file and the display time of each sub-image in the static PPT can be established, so as to synchronize the PPT content in the audio and video background from the time perspective, thereby avoiding the transition lag problem in the related technical solutions and further improving the smoothness of video playing.

[0076] In step 103, a PPT transition animation with a preset pause length is added between each sub-image to obtain an initial PPT video for the static PPT.

[0077] To make the transition between different PPT pages more natural, a PPT transition animation can be added between each sub-image, and the duration of the PPT transition animation is the same as the preset pause duration constructed in step 102, thereby obtaining an initial PPT video for the static PPT.

[0078] Illustratively, the play time of the PPT is matched with the time of each initial audio file, and a PPT transition animation is added, thereby forming a PPT video. That is, the initial audio file corresponding to each sub-image is obtained according to step 102, and the audio time of the initial audio file corresponding to each sub-image is recorded, the recorded audio time is mapped to the rehearsal time of each PPT, and an initial PPT video for the static PPT is synthesized. This process can strictly correspond the time when the virtual digital human narrates the PPT content and the time when the PPT transitions to the next PPT content, and the transition animation can be set to 1s (preset pause duration).

[0079] The following will be described in combination with Figure 4 The above embodiments are illustratively described. Figure 4 One of the PPT switching setting pages provided by an embodiment of the present application is shown in Figure 4 It is shown that parameters such as a single PPT page, an automatic switching speed of a PPT transition animation, and a switching mode are set.

[0080] As shown in Figure 4 After adding the transition animation of the PPT, the speed is set as the time of the PPT transition animation (i.e., the preset pause time), and the time of automatic film changing is the display time that can be set in each PPT page of the PPT. The display time needs to correspond to the audio time of the initial audio file of each sub-image, thereby ensuring the consistency of the PPT video time and the virtual digital human narration time. Based on the above setting operation, a complete initial PPT video of the PPT can be obtained.

[0081] In step 104, a virtual digital human is added to the initial PPT video, and the virtual digital human is controlled to comment on the first target text information in the initial PPT video based on the target audio file, thereby obtaining a target PPT video.

[0082] Illustratively, a virtual digital human can be added to the initial PPT video, thereby driving the virtual digital human to comment on the first target text information in the initial PPT video based on the target audio file, so that the mouth shape of the virtual digital human corresponds to the narrated text information, thereby realizing a target PPT video in which the virtual digital human narrates the PPT content.

[0083] For the virtual digital human, pre-selection can be performed, which will be described in combination with specific embodiments:

[0084] In an optional embodiment of the present disclosure, a target digital human image is determined from candidate digital human images; and a virtual digital human is constructed based on preconfigured digital human parameters and the target digital human image.

[0085] The digital human parameters at least include one or more of tone, posture, expression, and body movement. The candidate digital human images are a plurality of digital human images available for user selection. The digital human image refers to the facial features of a digital human. The digital human image is mainly of a 2D image type, thereby reducing the consumption of storage resources.

[0086] For example, the construction and selection of the virtual digital human can rely on a digital human technology platform, i.e., the digital human technology platform includes a plurality of candidate digital human images and provides selectable digital human parameters for each candidate digital human image. Since the digital human image is mainly of a 2D image type, it is composed of a specific posture, a specific body movement, and a specific expression, for example, a sitting posture or a standing posture, a hand lifting gesture, a nod, a smile, and seriousness. During video production, the video resolution can be selected; and videos are recorded by combining standing, sitting, smiling, surprise, and hand lifting gestures.

[0087] The following will be described in combination with Figure 5 Figure 5 Fig. 1 shows a schematic diagram of one of the candidate digital human images provided by an embodiment of the present disclosure; Figure 5 The three selectable digital human images shown are all in a standing posture, but differ in hand body movement and expression, for example, the first virtual digital human has a smiling expression, and the second virtual digital human has no expression. That is, in the digital human template system, the digital human image parameters such as movement and expression can be selected to edit the image of the virtual digital human. It can be understood that if the user directly uses the image mode of the digital human, the virtual digital human will perform the narration of the PPT content according to the default parameters. The audio file generated by the above embodiment can select different tones according to the image of the virtual digital human and in combination with female voice and male voice, to form different initial audio files.

[0088] It should be explained that Figure 5 The virtual digital human shown can be an animation cartoon character, a virtual human identical to the actual human image, or a virtual human formed by combining the two, and the present disclosure does not make any special limitation on this.

[0089] Further, after selecting the virtual digital human to be added, the virtual digital human can be controlled to comment on the first target text information in the initial PPT video based on the target audio file, to obtain a target PPT video.

[0090] ​In one optional embodiment of this disclosure, audio features are extracted from the target audio file; a three-dimensional facial mesh is generated for the virtual digital human, and projection is performed on the facial feature markers on the two-dimensional facial plane of the virtual digital human; the facial feature points in the three-dimensional facial mesh are offset based on the audio features and a diffusion model to narrate the first target text information in the initial PPT video, thereby obtaining the target PPT video.

[0091] For example, an audio-driven approach can be used to match the lip movements of a virtual digital human with the text, thereby creating animated videos of a digital human narrating PowerPoint content. Methods such as AniPortrait can be used to match the lip movements of the virtual digital human with the audio, driving the virtual digital human to speak based on a target audio file.

[0092] Specifically, the process begins with audio feature extraction, followed by 3D facial mesh and pose generation for the digital human avatar. Next, 2D markers are used to form a projection, and finally, a diffusion model is employed for animation generation. To improve the accuracy of lip movements, AniPortrait introduces the PoseGuider module, which uses ControlNet's multi-scale strategy to integrate marker features of different scales into different layers of the network. Additionally, markers from a reference image are included as extra input, and a cross-attention mechanism further enhances the accuracy of the generated animation.

[0093] Figure 6 This application provides a schematic diagram of a video file illustrating an audio-driven virtual digital human presenting a static PowerPoint presentation, as one embodiment of the present application; see reference. Figure 6 As shown, with Figure 5 Taking the first virtual digital human character as an example, combining the virtual digital human character with the target audio file can yield... Figure 6 The video file shown.

[0094] Furthermore, use the PPT video as a background and place the aforementioned elements in appropriate positions on the screen. Figure 6 The virtual digital human narrating the animation shown can be synthesized as follows: Figure 7 The target PPT video is shown.

[0095] Reference Figure 7 As shown, the above embodiment yields a smooth PPT video (i.e., the initial PPT video), which is then used to obtain... Figure 5 The animation shown depicts a virtual digital human presenting PowerPoint content, which can be like... Figure 7As shown, the initial PPT video is synthesized as a background with the virtual digital person telling the animation, and the virtual digital person can be placed at the lower left or lower right position of the initial PPT video background, so as to form the target PPT video of the virtual digital person telling the PPT content. In this process, attention should be paid to the correspondence between the PPT video time and the digital person telling time.

[0096] Through the above embodiment, the method of using pre-trained model and artificial intelligence can make the conversion of text to audio and the digital person telling the text become simple and easy to operate.

[0097] In an optional embodiment of the present application, before the step 101 of extracting the first target text information in each sub-image from the sequence image set corresponding to the static PPT, the method further comprises performing HDR (High Dynamic Range) image reconstruction on each sub-image to improve the image quality of each sub-image, and the reconstruction process at least includes:

[0098] obtaining each sub-image and performing feature extraction on the sub-image to obtain an attention information feature map for the sub-image;

[0099] inputting the attention information feature map into a pre-trained image reconstruction model to obtain a reconstructed high dynamic range (HDR) image; wherein the image reconstruction model at least includes a local enhancement module and a spatial feature transformation module, the local enhancement module is used to extract corresponding local spatial information from the attention information feature map, and the spatial feature transformation module is used to perform spatial affine transformation on the local spatial information to obtain a HDR image corresponding to each sub-image;

[0100] extracting the first target text information in each sub-image based on each HDR image.

[0101] In an optional embodiment of the present application, the feature extraction on the sub-image to obtain the attention information feature map for the single-frame sub-image comprises:

[0102] inputting the sub-image into a pre-trained Transformer model to extract a plurality of deep local feature representations from the sub-image based on the Transformer model, determining the relative positions of each of the deep local feature representations, and generating relative position encodings of each of the deep local feature representations;

[0103] inputting the plurality of deep local feature representations and the relative position encodings of each of the deep local feature representations into a pre-trained SwinIR model for feature extraction to obtain the attention information feature map for the sub-image.

[0104] In an optional embodiment of the present application, the inputting of the attention information feature map into the pre-trained image reconstruction model to obtain the reconstructed high dynamic range (HDR) image comprises:

[0105] The attention information feature map is processed by dimension reduction to convert the image size of the attention information feature map to the same size as that of the sub-image.

[0106] The attention information feature map processed by dimension reduction is inputted into the image reconstruction model to obtain the reconstructed HDR image.

[0107] In an optional embodiment of the present application, the local enhancement module comprises at least a residual block structure, and the residual block structure comprises a dense connection layer, a local feature fusion layer and a local residual learning layer, which are respectively used to extract corresponding local spatial information from the attention information feature map.

[0108] In an optional embodiment of the present application, the local enhancement module further comprises a UNet network structure, and the UNet network structure is arranged before the residual block structure to reconstruct an image based on the UNet network structure through the attention information feature map.

[0109] In an optional embodiment of the present application, the method further comprises:

[0110] The local spatial information is subjected to spatial affine transformation according to the spatial feature transformation module in the image reconstruction model to obtain image features of an HDR image.

[0111] The image features of the HDR image are subjected to feature fusion to obtain an initial HDR image.

[0112] The initial HDR image is subjected to tone mapping to obtain a reconstructed HDR image.

[0113] In an optional embodiment of the present application, the tone mapping of the initial HDR image to obtain the reconstructed HDR image comprises:

[0114] The initial HDR image is subjected to tone mapping based on a Gamma tone mapping function or a μ-Law tone mapping function to obtain the reconstructed HDR image.

[0115] In an optional embodiment of the present application, before the step 101, the method further comprises auditing the graphic content of each sub-image, and the auditing process comprises:

[0116] The pre-configured unstructured data processing module utilizes a multi-modal large model to perform semantic analysis on unstructured data such as news text, audio, video, etc. in each sub-image, generates a semantic correlation graph, extracts core context and information correlation in text and images through deep analysis, and constructs a preliminary review dataset;

[0117] The pre-configured multi-modal data integration module integrates the text data, audio data and video data in each sub-image in a unified context, converts them into a text correlation vector space, and transmits them to the content analysis module for deep analysis of content context and emotion;

[0118] The content analysis module is configured to preliminarily determine the matching degree of text and image content based on the semantic correlation graph generated by the unstructured data processing module, generate a context factor Qycs, and analyze potential sensitive content in each modal data to generate a sensitive factor Gmyz. Then, based on the context factor Qycs and the sensitive factor Gmyz, the potential emotional factors, social sensitive information and potential misleading factors in the text content are analyzed in layers to generate a multi-modal correlation factor Mmgyz.

[0119] The content anomaly quantification module calculates a review weight value Hqwz based on the context factor Qycs, the sensitive factor Gmyz and the multi-modal correlation factor Mmgyz. Then, by calculating the system review risk factor Rxz and associating it with the review weight value Hqwz, the final content anomaly index Cxzs is generated.

[0120] The calculation formula of the review weight value Hqwz is:

[0121] Hqwz = b1 x Qyca + b2 x Gmyz + b3 x Mmgyz + B;

[0122] Wherein, b1, b2 and b3 represent the preset weight coefficients of the context factor Qycs, the sensitive factor Gmyz and the multi-modal correlation factor Mmgyz, respectively, and B represents the review correction coefficient.

[0123] The intelligent control module is configured to set a compliance threshold T and a review risk threshold R, compare the review weight value Hqwz with the compliance threshold T to determine whether the current content meets the review standard, compare the review risk threshold R with the content anomaly index Cxzs to obtain the final review result, and selectively trigger the control scheme of manual review or automatic release according to the review result.

[0124] In an optional embodiment of the present application, the unstructured data processing module is configured to first receive the unstructured data of the text in each sub-image for format standardization processing, including image resolution adjustment and text language normalization processing.

[0125] Then the pre-training network of the multi-modal large model is called to extract multi-level features including visual features, semantic features and emotional features from the normalized unstructured data in each sub-image, and the extracted multi-level features are normalized to generate a single-modal feature matrix; based on the single-modal feature matrix, the multi-level features of the images and texts in each sub-image are mapped in a multi-modal correlation vector space, and a cross-modal semantic correlation graph is generated through the semantic embedding mechanism of the multi-modal large model.

[0126] The cross-modal semantic correlation graph is used to analyze the deep correlation between the text and the sub-images, extract core context elements in the text and images, and simultaneously analyze the emotional state and expression trend, so as to identify potential sensitive information, misleading information and social influence factors in the image-text content in real time, and form a multi-modal correlation semantic index.

[0127] Finally, a preliminary review data set is generated.

[0128] In an optional embodiment of the present application, the method further comprises: calling the multi-modal data integration module to receive the preliminary review data set generated by the unstructured data processing module, and performing time sequence and content alignment processing on the image-text data, audio data and video data therein through a frame synchronization algorithm.

[0129] A multi-modal feature mapping algorithm is used to fuse the image-text context, emotion and semantic features into a multi-modal feature vector, and an association degree matrix is generated according to the association degree between each modality to perform hierarchical mapping on information density, context relevance and semantic consistency.

[0130] Based on the association degree matrix, an image-text correlation vector space is constructed to quantize the multi-modal data in a unified semantic space to generate an integrated semantic correlation representation vector, and a context adjustment algorithm is used to correct the semantic deviation caused by modality differences.

[0131] The integrated image-text correlation vector space is transmitted to the content analysis module for analysis.

[0132] In an optional embodiment of the present application, by analyzing the semantic similarity and context consistency between the image and the text, semantic deviation related data and content matching related data are identified; the semantic similarity Ycy, the context consistency Sjy, the context correlation deviation Qjl and the image-text emotional consistency Twq in the semantic deviation related data and the content matching related data are extracted and dimensionless processed, and then a context factor Qycs is calculated through the following formula:

[0133]

[0134] Wherein, the semantic similarity Ycy is used to measure the similarity of the content of the text and the content expressed by the image in the semantic level, and the text vector representation is obtained by inputting the text into the pre-trained multi-modal model; the context consistency Sjy mainly measures the consistency degree of the multi-dimensional information such as time, place, person or event between the text context and the image context; the context correlation deviation degree Qjl is used to quantify the potential context mismatching degree between the text and the image, and after aligning the text and the image, the part where the "core scene or main object" is inconsistent is identified, and the inconsistent categories or quantity are counted, so as to obtain; the text-image emotion consistency Twq is used to measure the consistency degree between the emotion conveyed by the text and the emotion conveyed by the image picture, and the text emotion distribution and the image emotion distribution are calculated to obtain the similarity;

[0135] By comparing the preset context threshold Q with the context factor Qycs, the context correlation of the text and the image in the review scene is evaluated, and the specific comparison and evaluation contents are as follows:

[0136] If the context factor Qycs is greater than or equal to the context threshold Q, it indicates that the context of the text and the image is matched, and the semantics, emotion and context of the two are consistent, and the review is passed and marked as "context correlation is normal";

[0137] If the context factor Qycs is less than the context threshold Q, it indicates that the context of the text and the image is not matched, and the semantics, emotion and context of the two are not consistent, and the review is not passed and the specific deviation reason is further analyzed, including potential sensitive content.

[0138] In an optional embodiment of the present application, the multi-layer sensitivity analysis unit is used to further analyze the potential sensitive content of the text-image data, including analyzing the emotion-related data, social sensitivity-related data and potential misleading-related data in the text and the image, and after the emotion-related data, social sensitivity-related data and potential misleading-related data in the text and the image are analyzed, summarized and dimensionless processed, a sensitive content data set is constructed; the sensitive content data set is extracted to generate a sensitive factor Gmyz by the following formula:

[0139]

[0140] In the formula, Qmq represents the emotion intensity coefficient in the sensitive content data set, Mgc represents the sensitive word index in the sensitive content data set, and Xwg represents the information misleading possibility in the sensitive content data set.

[0141] In an optional embodiment of the present application, the correlation factor generation unit is configured to quantify the correlation degree between the sentiment, context and sensitive content by fusing multi-level information in the image-text correlation vector space based on the context factor Qycs and the sensitive factor Gmyz, and to generate the multi-modal correlation factor Mmgyz by the following formula:

[0142]

[0143] In an optional embodiment of the present application, the content anomaly quantification module comprises an audit calculation unit and an anomaly index acquisition unit.

[0144] The content evaluation unit is configured to construct a system audit risk factor Rxz, to obtain a semantic deviation degree Ycp and a content consistency index Nry by extracting semantic deviation related data and content matching related data, and to obtain a high sensitivity trigger rate Gmg by extracting a sensitive content data set, and to calculate and obtain the system audit risk factor Rxz in combination with the following formula:

[0145]

[0146] In an optional embodiment of the present application, the anomaly index acquisition unit is configured to calculate and obtain a final content anomaly index Cxzs, and the specific calculation formula is as follows:

[0147]

[0148] In an optional embodiment of the present application, the intelligent control module compares and evaluates the audit weight value Hqwz with the compliance threshold T and compares and evaluates the audit risk threshold R with the content anomaly index Cxzs, and the following evaluation contents are generated in detail:

[0149] Compliance standard comparison:

[0150] If the audit weight value Hqwz is greater than or equal to the compliance threshold T, it indicates that the content meets the audit standard, and the system is marked as “compliant”.

[0151] If the audit weight value Hqwz is less than the compliance threshold T, it indicates that the content does not meet the audit standard, and there is a compliance deficiency, which needs to be further reviewed or adjusted.

[0152] Abnormal risk evaluation:

[0153] If the content anomaly index Cxzs is greater than or equal to the audit risk threshold R, it indicates that the content has an abnormal risk, and the system generates an “abnormal content” mark to trigger an artificial review process.

[0154] If the content anomaly index Cxzs is less than the audit risk threshold R, it indicates that the content does not have an abnormal risk, and the system generates a “normal content” mark, and enters an automatic publishing process.

[0155] When the audit weight value Hqwz and the evaluation result of the content anomaly index Cxzs obtain the labels of "compliance" and "normal content" at the same time, the content is automatically labeled as "pass" and directly published;

[0156] When the audit weight value Hqwz and the evaluation result of the content anomaly index Cxzs are content that does not meet the audit standard or has an abnormal risk, the content is automatically labeled as "to be reviewed", and a selective review process is triggered.

[0157] A control scheme for setting the compliance threshold T and the audit risk threshold R, judging whether the current content meets the audit standard, obtaining the final audit result, and selectively triggering manual review or automatic publishing according to the audit result.

[0158] It should be understood that although each step in the flowchart is displayed in sequence according to the direction of the arrow, these steps are not necessarily executed in the order indicated by the arrow. Unless explicitly stated in this article, the execution of these steps has no strict order restriction, and these steps can be executed in other orders. Moreover, at least part of the steps in the figure can include multiple sub-steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily sequential, but can be executed with at least part of other steps or other steps. Sub-steps or stages of the step or phase are executed in rotation or alternation.

[0159] In order to realize the video production method of the digital person automatically explaining the static PPT, please refer to Figure 8 An embodiment of the present application provides a video production device 800 for a digital person automatically explaining a static PPT. The video production device 800 for the digital person automatically explaining the static PPT can include a text extraction module 801, an audio synthesis module 802, an animation adding module 803 and a control module 804.

[0160] The text extraction module 801 is configured to extract first target text information in each sub-image from a sequence image set of the static PPT.

[0161] In an optional embodiment of the present disclosure, the text extraction module 801 is specifically configured to obtain the static PPT; store the static PPT in an image format to obtain a sequence image set corresponding to the static PPT; or perform a screenshot operation on each PPT page in the static PPT to obtain the sequence image set corresponding to the static PPT; or process the static PPT based on a preset conversion model or a class file to obtain the sequence image set corresponding to the static PPT; and extract the first target text information in each sub-image from the sequence image set of the static PPT.

[0162] In an optional embodiment of the present disclosure, the text extraction module 801 is specifically configured to extract initial text information in each sub-image from the sequence image set corresponding to the static PPT; and perform content expansion on the initial text information based on an image-text multi-modal understanding model to obtain the first target text information in each sub-image.

[0163] In an optional embodiment of the present disclosure, the audio synthesis module 802 is configured to add a preset pause duration after each initial audio file corresponding to each sub-image; and synthesize the initial audio file corresponding to each sub-image and the preset pause duration to obtain the target audio file for the static PPT.

[0164] In an optional embodiment of the present disclosure, the audio synthesis module 802 can also be configured to integrate the first target text information in each sub-image into the same text to obtain second target text information after integration; add a preset pause duration after the end text mark of the first target text information of each sub-image in the second target text information; and convert the second target text information and the preset pause duration into speech to obtain the target audio file for the static PPT.

[0165] In an optional embodiment of the present disclosure, the apparatus can further include an image determining module configured to determine a target digital human image from the candidate digital human images; and a digital human constructing module configured to construct a virtual digital human according to the preconfigured digital human parameters and the target digital human image, wherein the digital human parameters include at least one of tone, posture, expression, and limb movement.

[0166] In an optional embodiment of the present disclosure, the control module is configured to extract audio features from the target audio file; generate a three-dimensional face mesh for the virtual digital human, and project the face feature points of the three-dimensional face mesh based on the two-dimensional face plane of the virtual digital human; and control the face feature points in the three-dimensional face mesh to shift based on the audio features and the diffusion model, so as to comment on the first target text information in the initial PPT video and obtain the target PPT video.

[0167] For specific limitations of the above-mentioned video production apparatus for automatically explaining static PPT by digital human, reference can be made to the limitations of the method for automatically explaining static PPT by digital human as described above, which will not be repeated here. Each module in the above-mentioned video production apparatus for automatically explaining static PPT by digital human can be realized by software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or stored in the memory in the computer device in software form, so as to be called and executed by the processor to perform the operations corresponding to each module.

[0168] In an embodiment, a computer device is provided, and an internal structure diagram of the computer device can be as shown in Figure 9 The computer device includes a processor, a memory, a network interface, and a database connected by a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The database of the computer device is configured to store data. The network interface of the computer device is configured to communicate with an external terminal through a network connection. The computer program is executed by the processor to implement the above-mentioned method for automatically explaining static PPT by digital human. The computer program includes a memory and a processor, and the memory stores a computer program. When the processor executes the computer program, it implements any step in the above-mentioned method for automatically explaining static PPT by digital human.

[0169] In one embodiment, a computer readable storage medium is provided, having stored thereon a computer program which, when executed by a processor, can implement any step of the method for automatically making a video of a digital human explaining a static PPT as above.

[0170] Those skilled in the art will appreciate that embodiments of the application can be supplied as a method, a system, or a computer program product. Accordingly, the application can be embodied in the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the application can be embodied in the form of a computer program product on one or more computer readable storage media (including, but not limited to, disk memory, CD-ROMs, optical storage media, etc.) having computer usable program code embodied thereon.

[0171] The present application is described in reference to the flowchart illustrations and / or block diagrams according to the embodiments of the application. It is understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0172] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks. Figure 1 one or more functions specified in the flowchart illustrations and / or block diagrams block or blocks.

[0174] While the preferred embodiments of the application have been described, additional variations and modifications can be made to these embodiments by those skilled in the art once they have the benefit of the present disclosure without departing from the spirit and scope of the application. Accordingly, it is intended that such additions and modifications be included within the scope of the application. It is the following claims, including any amendments thereto, which define the scope of the application.

[0175] Obviously, numerous modifications and variations of the present application are possible in light of the above teachings. It is therefore to be understood that within the scope of the appended claims and their equivalents, the application can be practiced otherwise than as specifically described.

Claims

1. A method for creating videos of a digital human automatically explaining a static PPT, characterized in that, include: Each sub-image in a static PPT is obtained, and the sub-images are input into a pre-trained Transformer model to extract multiple deep local feature representations from the sub-images based on the Transformer model. The relative positions of each deep local feature representation are determined, and a relative position encoding of each deep local feature representation is generated. The multiple deep local feature representations and their relative position encodings are input into a pre-trained SwinIR model for feature extraction to obtain an attention information feature map for the sub-images. The attention information feature map is input into a pre-trained image reconstruction model to obtain a reconstructed high dynamic range (HDR) image. The image reconstruction model includes at least a local enhancement module and a spatial feature transformation module. The local enhancement module is used to extract corresponding local spatial information from the attention information feature map, and the spatial feature transformation module is used to perform spatial affine transformation on the local spatial information to obtain the HDR image corresponding to each sub-image. Extract the first target text information from each corresponding sub-image based on each HDR image; The first target text information in each of the sub-images is converted into speech to obtain an initial audio file corresponding to each sub-image. The initial audio files corresponding to each sub-image are then combined to obtain a target audio file for the static PPT. A preset pause duration is added between each of the initial audio files in the target audio file, and a correspondence is established between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT. Add the preset pause duration PPT transition animation between each of the sub-images to obtain the initial PPT video for the static PPT; Adding a virtual digital human to the initial PPT video and controlling the virtual digital human to narrate the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video includes: extracting audio features from the target audio file; generating a three-dimensional facial mesh for the virtual digital human and projecting it based on facial feature markers on the two-dimensional facial plane of the virtual digital human; and controlling the facial feature points in the three-dimensional facial mesh to offset based on the audio features and a diffusion model to narrate the first target text information in the initial PPT video to obtain the target PPT video.

2. The method according to claim 1, characterized in that, The step of extracting the first target text information from each corresponding sub-image based on each HDR image includes: The static PPT can be stored as an image to obtain a sequence of images corresponding to the static PPT; or, screenshots can be taken of each PPT page in the static PPT to obtain a sequence of images corresponding to the static PPT; or, the static PPT can be processed based on a preset conversion model or class file to obtain a sequence of images corresponding to the static PPT. Extract the first target text information from each sub-image of the sequence of images in the static PPT.

3. The method according to claim 2, characterized in that, The step of extracting the first target text information from each sub-image of a static PPT sequence image set includes: Extract the initial text information from each sub-image from the sequence image set corresponding to the static PPT; The initial text information is expanded based on the image-text multimodal understanding model to obtain the first target text information for each sub-image.

4. The method according to claim 1, characterized in that, The initial audio files corresponding to each of the sub-images are combined to obtain the target audio file for the static PPT, including: adding the preset pause duration to the initial audio files corresponding to each of the sub-images respectively; The initial audio files corresponding to each of the sub-images and the preset pause duration are combined to obtain the target audio file for the static PPT.

5. The method according to claim 1, characterized in that, The method further includes: The first target text information in each of the sub-images is integrated into the same text to obtain the integrated second target text information; In the second target text information, the preset pause duration is added after the end text identifier of the first target text information of each sub-image; The second target text information and the preset pause duration are converted into speech to obtain the target audio file for the static PPT.

6. The method according to claim 1, characterized in that, The method further includes: Determine the target digital human image from the candidate digital human images; The virtual digital human is constructed based on pre-configured digital human parameters and the target digital human image; the digital human parameters include at least one or more of the following: timbre, posture, facial expression, and body movements.

7. A video production device for automatically creating static PPT presentations using a digital human, characterized in that, include: The text extraction module is used to acquire each sub-image in a static PPT, input the sub-image into a pre-trained Transformer model to extract multiple deep local feature representations from the sub-image based on the Transformer model, determine the relative position of each deep local feature representation, and generate a relative position encoding for each deep local feature representation; input the multiple deep local feature representations and the relative position encoding of each deep local feature representation into a pre-trained SwinIR model for feature extraction to obtain an attention information feature map for the sub-image; The attention information feature map is input into a pre-trained image reconstruction model to obtain a reconstructed high dynamic range (HDR) image. The image reconstruction model includes at least a local enhancement module and a spatial feature transformation module. The local enhancement module is used to extract corresponding local spatial information from the attention information feature map, and the spatial feature transformation module is used to perform spatial affine transformation on the local spatial information to obtain the HDR image corresponding to each sub-image. Extract the first target text information from each corresponding sub-image based on each HDR image; An audio synthesis module is used to convert the first target text information in each of the sub-images into speech to obtain an initial audio file corresponding to each sub-image, and to synthesize the initial audio files corresponding to each of the sub-images to obtain a target audio file for the static PPT; wherein, a preset pause duration is added between each of the initial audio files in the target audio file, and a correspondence is established between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT; An animation adding module is used to add PPT transition animations with preset pause durations between each of the sub-images to obtain an initial PPT video for the static PPT; A control module is used to add a virtual digital human to the initial PPT video and control the virtual digital human to narrate the first target text information in the initial PPT video based on the target audio file, thereby obtaining a target PPT video. The module includes: extracting audio features from the target audio file; generating a three-dimensional facial mesh for the virtual digital human and projecting it based on facial feature markers on the two-dimensional facial plane of the virtual digital human; and controlling the offset of facial feature points in the three-dimensional facial mesh based on the audio features and a diffusion model to narrate the first target text information in the initial PPT video, thereby obtaining the target PPT video.

8. A computer device, comprising: The device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the video production method for automatically explaining static PPT by a digital human as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the steps of the video production method for automatically explaining static PPTs by a digital human, as described in any one of claims 1 to 6.

Images