Video production method for automatically explaining static PPT by digital person

Abstract

The invention provides a video production method and device for automatically explaining static PPT by a digital human, computer equipment and a computer storage medium, and relates to the technical field of video processing. Comprising the following steps: extracting first target text information in each sub-image from a sequence image set of static PPT; converting the first target text information in each subimage into voice, and synthesizing the obtained initial audio files corresponding to each subimage to obtain a target audio file for the static PPT; wherein a preset pause duration is added between the initial audio files in the target audio file, and a corresponding relation between the audio time of each initial audio file and the display time of each sub-image is constructed; adding a PPT transition animation with a preset pause duration between the sub-images to obtain an initial PPT video for the static PPT; and based on the target audio file, controlling the virtual digital human to explain first target text information in the initial PPT video to obtain a target PPT video. According to the method, the video playing fluency can be improved.

Description

Technical Field

[0001] The present application relates to the field of video processing technology, and in particular to a method for producing a video of a digital human automatically explaining a static PPT, a device for producing a video of a digital human automatically explaining a static PPT, a computer device, and a computer storage medium. Background Art

[0002] With the continuous development of the Internet and virtual technology, there are more and more application scenarios for virtual digital humans, including a scenario of converting PPT into video and broadcasting the PPT content through virtual digital humans. However, the pictures in PPT are static pictures, so transition animations need to be added in the process of converting PPT into video to achieve a smooth transition effect. Otherwise, there will be transition jams, which will affect the smoothness of the video production of the digital human automatically explaining the PPT.

[0003] In the related technical solutions, the PPT image is usually used as the background, and the audio is used to drive the digital human to tell a short animation of the PPT so that each PPT is synthesized into a short video, and finally synthesized into a complete explanation video. However, the above process still has the problem of video freeze caused by PPT transition, which in turn affects the smoothness of video playback. Summary of the invention

[0004] The embodiments of the present application provide a method for producing a video of a digital human automatically explaining a static PPT, a device for producing a video of a digital human automatically explaining a static PPT, a computer device, and a computer storage medium, thereby overcoming, at least to a certain extent, the technical problem that, due to the limitations and defects of the relevant technology, a video produced based on a static PPT is played, resulting in video freezes caused by PPT transitions, thereby affecting the smoothness of video playback.

[0005] A first aspect of an embodiment of the present application provides a method for producing a video in which a digital human automatically explains a static PPT, the method comprising: extracting first target text information in each sub-image from a set of sequential images of the static PPT; converting the first target text information in each sub-image into speech to obtain an initial audio file corresponding to each sub-image, and synthesizing the initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of each sub-image corresponding to the static PPT is constructed; a PPT transition animation with a preset pause duration is added between each sub-image to obtain an initial PPT video for the static PPT; a virtual digital human is added to the initial PPT video, and the virtual digital human is controlled to explain the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video.

[0006] According to a second aspect of an embodiment of the present application, there is provided a video production device in which a digital human automatically explains a static PPT, the device comprising: a text extraction module, for extracting first target text information in each sub-image from a set of sequential images of the static PPT; an audio synthesis module, for converting the first target text information in each sub-image into speech, obtaining an initial audio file corresponding to each sub-image, and synthesizing the initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of each sub-image corresponding to the static PPT is constructed; an animation adding module, for adding a PPT transition animation with a preset pause duration between each sub-image, to obtain an initial PPT video for the static PPT; and a control module, for adding a virtual digital human to the initial PPT video, and controlling the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file, to obtain the target PPT video.

[0007] According to a third aspect of the embodiments of the present application, a computer device is provided, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any of the above methods for producing a video in which a digital human automatically explains a static PPT are implemented.

[0008] According to a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for producing a video of a digital human automatically explaining a static PPT as described in any of the above items are implemented.

[0009] According to a fifth aspect of the embodiments of the present application, a computer program product is provided, including a computer program, which is executed by a processor to implement any of the steps of the above method for producing a video of a digital human automatically explaining a static PPT.

[0010] The technical solution of this application has the following beneficial effects:

[0011] A method for producing a video of a digital human automatically explaining a static PPT comprises extracting the first target text information in each sub-image from a set of sequential images of the static PPT; converting the first target text information in each sub-image into speech to obtain an initial audio file corresponding to each sub-image, and synthesizing the initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT; wherein a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of each sub-image corresponding to the static PPT is constructed; adding a PPT transition animation with a preset pause duration between each sub-image to obtain an initial PPT video for the static PPT; adding a virtual digital human to the initial PPT video, and controlling the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file to obtain a target PPT video. On the one hand, the method automatically converts the text information corresponding to each sub-image obtained from the static PPT into audio, and drives the virtual digital human to automatically explain through audio, so that the text information corresponding to each sub-image corresponds to the content explained by the virtual digital human, thereby improving the accuracy of the static PPT explanation. On the other hand, a correspondence between the audio time of the initial audio file and the display time of each corresponding sub-image in the static PPT is constructed, and based on the target audio file, the virtual digital human is controlled to explain the first target text information in the initial PPT video, thereby ensuring that the PPT animation displayed in the background and the video animation explained by the virtual digital human are synchronized and combined together through the time relationship, thereby avoiding the jamming problem caused by the abrupt PPT transition, thereby ensuring the smoothness of the video playback. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0013] Figure 1 A flowchart of a method for producing a video of a digital human automatically explaining a static PPT provided in one embodiment of the present application;

[0014] Figure 2 A schematic diagram of a first target text information generated according to an embodiment of the present application;

[0015] Figure 3 A schematic diagram of a synthetic target audio file provided by one embodiment of the present application;

[0016] Figure 4 A schematic diagram of a PPT switching setting page provided in one embodiment of the present application;

[0017] Figure 5 A schematic diagram of a candidate digital human image provided in one embodiment of the present application;

[0018] Figure 6 A schematic diagram of a video file of an audio-driven virtual digital human narrating a static PPT provided in one embodiment of the present application;

[0019] Figure 7 A schematic diagram of a target PPT video provided in one embodiment of the present application;

[0020] Figure 8 A schematic diagram of the structure of a video production device for automatically explaining a static PPT using a digital human provided in one embodiment of the present application;

[0021] Fig. 9 A schematic diagram of the computer device structure provided for one embodiment of the present application. DETAILED DESCRIPTION

[0022] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as being limited to the examples set forth herein; on the contrary, these embodiments are provided so that the present disclosure will be more comprehensive and complete, and the concepts of the example embodiments are fully conveyed to those skilled in the art. The described features, structures, or characteristics may be combined in one or more embodiments in any suitable manner. In the following description, many specific details are provided to provide a full understanding of the embodiments of the present disclosure. However, those skilled in the art will appreciate that the technical solutions of the present disclosure may be practiced while omitting one or more of the specific details, or other methods, components, devices, steps, etc. may be adopted. In other cases, known technical solutions are not shown or described in detail to avoid obscuring various aspects of the present disclosure.

[0023] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0024] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0025] In the relevant technical background, with the continuous development of the Internet and virtual technology, the application scenarios of virtual digital people are increasing, thus deriving into artificial intelligence (AI) virtual image products in various fields. For example, a scenario in which PPT is converted into a video and the PPT content is broadcast by a virtual digital person. In the existing AI virtual image products, the usual operation process is anchor video acquisition, data processing, model training, and image output; among them, the production output stage is based on the trained anchor image, and the video is generated and output according to the input manuscript, voice, optional anchor emotions, background pictures, videos, standing posture, sitting posture, etc. However, in the current AI digital human video generation system, data flows in one direction, and only pictures and videos can be input as backgrounds to be synthesized with AI digital people. In the above application scenarios, the characteristics of static PPT are similar to still pictures, and in the process of converting PPT to video, only by adding transition animations can the transition be smooth, otherwise there will be transition jams, which will affect the smoothness of the video production of virtual digital people automatically explaining PPT.

[0026] In the existing technical method, the PPT file is converted into an image file, and the audio is used to drive the virtual digital human to tell the content of each PPT to form a short animation, and finally the above multiple short videos are combined into a complete video. However, the video produced by the above method is still directly converted into a video like a static image, which will cause technical problems such as transition jams, thereby affecting the smoothness of the video of the virtual digital human automatically explaining the PPT content.

[0027] In view of the above technical problems, the embodiment of the present application proposes a video production method for a digital human to automatically explain a static PPT. The method can be applied to any application scenario that requires the use of a virtual digital human for video production. The method automatically converts the text information corresponding to each sub-image obtained from the static PPT into audio, and drives the virtual digital human to automatically explain through audio, so that the text information corresponding to each sub-image corresponds to the content explained by the virtual digital human, thereby improving the accuracy of the static PPT explanation. On the other hand, the corresponding relationship between the audio time of the initial audio file and the display time of each sub-image in the static PPT is constructed, and the virtual digital human is controlled based on the target audio file to explain the first target text information in the initial PPT video, thereby ensuring that the PPT animation displayed in the background is synchronized with the video animation explained by the virtual digital human, so as to be combined together through the time relationship, thereby avoiding the jamming problem caused by the abrupt PPT transition, thereby ensuring the smoothness of the video playback.

[0028] The following method may be executed by a terminal device or a server, or may be executed by a terminal device and a server together. Here, the method is specifically illustrated by taking a terminal device or a server as the execution subject as an example. Figure 1 This is a flowchart of a method for producing a video of a digital human automatically explaining a static PPT provided in one embodiment of the present application, referring to Figure 1 As shown, it includes the following steps 101 to 104:

[0029] Step 101: extract the first target text information in each sub-image from a sequence image set corresponding to a static PPT.

[0030] Step 102: convert the first target text information in each sub-image into speech to obtain an initial audio file corresponding to each sub-image, and synthesize the initial audio files corresponding to each sub-image to obtain a target audio file for the static PPT.

[0031] Among them, a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT is constructed.

[0032] Step 103: Add a PPT transition animation with a preset pause duration between each sub-image to obtain an initial PPT video for the static PPT.

[0033] Step 104: Add a virtual digital human to the initial PPT video, and control the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video.

[0034] Through the above Figure 1 In the embodiment shown, on the one hand, the method automatically converts the text information corresponding to each sub-image obtained from the static PPT into audio, and drives the virtual digital human to perform automatic interpretation through audio, so that the text information corresponding to each sub-image corresponds to the content of the virtual digital human interpretation, thereby improving the accuracy of the static PPT interpretation. On the other hand, the corresponding relationship between the audio time of the initial audio file and the display time of each sub-image in the static PPT is constructed, and the virtual digital human is controlled based on the target audio file to interpret the first target text information in the initial PPT video, thereby ensuring that the PPT animation displayed in the background and the video animation of the virtual digital human interpretation are synchronized, so as to be combined together through the time relationship, thereby avoiding the jamming problem caused by the abrupt PPT transition, thereby ensuring the smoothness of video playback.

[0035] The above-mentioned Figure 1 The various steps shown in are described in detail.

[0036] In step 101, first target text information in each sub-image is extracted from a sequence image set corresponding to a static PPT.

[0037] The static PPT includes multiple static PPT pages. Exemplarily, the first target text information in each sub-image can be extracted from the corresponding sequence image set in the multiple static PPT pages.

[0038] When executing the above step 101, since the static PPT contains multiple static PPT pages, which are not image files, it is necessary to first convert the multiple static PPT pages into image files to obtain multiple seed images corresponding to the multiple static PPT pages, and obtain a sequence image set. The following will be described in detail in conjunction with a specific embodiment:

[0039] Embodiment 1: In an optional embodiment of the present disclosure, a static PPT is obtained; the static PPT is stored in an image format to obtain a set of sequential images corresponding to the static PPT; and the first target text information in each sub-image is extracted from the set of sequential images corresponding to the static PPT.

[0040] Exemplarily, the PPT file may be directly saved as an image file to implement the conversion process between the PPT file and the image file.

[0041] Embodiment 2: In an optional embodiment of the present disclosure, a screenshot operation is performed on each PPT page in a static PPT to obtain a sequence image set corresponding to the static PPT; and the first target text information in each sub-image is extracted from the sequence image set corresponding to the static PPT.

[0042] Exemplarily, screenshots may be taken of multiple PPT pages in a PPT file to obtain corresponding images, thereby obtaining a sequence image set corresponding to the static PPT.

[0043] Embodiment 3: In an optional embodiment of the present disclosure, a static PPT is processed based on a preset conversion model or class file to obtain a set of sequence images corresponding to the static PPT; and the first target text information in each sub-image is extracted from the set of sequence images corresponding to the static PPT.

[0044] Exemplarily, the image conversion operation of the static PPT can be implemented based on a pre-trained preset conversion model; or the image conversion operation of the static PPT can be implemented using the corresponding class file. For example, JAVA can be used to import the relevant class files in the Apache POI library to process the static PPT to obtain a sequence of images.

[0045] Through the above embodiment, the PPT page is converted into an image file, which facilitates the subsequent extraction of text information in the image, so as to accurately and quickly obtain the first target text information in each sub-image, thereby improving the accuracy of the virtual digital human's explanation of the first target text information in each sub-image.

[0046] After the sequence image set corresponding to the presentation PPT is obtained based on the above embodiment, the first target text information in each sub-image can be extracted from the sequence image set corresponding to the static PPT.

[0047] In an optional embodiment of the present disclosure, initial text information in each sub-image is extracted from a set of sequential images corresponding to a static PPT; the initial text information is expanded based on an image-text multimodal understanding model to obtain first target text information for each sub-image.

[0048] The image-text multimodal understanding model is a large model used to understand the image content and expand the image content. For example, the image-text multimodal understanding model can be a CogVLM multimodal large model, a Tongyi Qianwen model, a Wenxin Yiyan model, etc., and the embodiments of the present application do not impose any restrictions on this.

[0049] Exemplarily, the initial text information is text proposed based on the text information contained in each sub-image, and the first target text information is text information obtained by expanding the content of the initial text information based on the image-text multimodal understanding model. When explaining the PPT, it is too stiff and dull to only explain the initial text information in the above static PPT. In order to be closer to the actual explanation needs, the extracted initial text information can be expanded.

[0050] For ease of understanding, the following will be combined Figure 2 The above embodiment is described exemplarily based on the first target text information in each sub-image.

[0051] Figure 2 A schematic diagram of a first target text information generated according to an embodiment of the present application is provided, referring to Figure 2 The graphics and text content displayed in the PPT page shown in the upper part can only obtain the initial text information such as "Table of Contents" and "1. Work Overview; 2. Completion Status; 3. Achievement Display; 4. Insufficient Work; 5. Work Plan". The initial text information only contains prompt words. After the CogVLM multimodal large model expands the content of the above prompt words, it can be obtained. Figure 2 The first target text information in the middle and lower part is used so that the virtual digital human can explain the static PPT based on the first target text information.

[0052] Depend on Figure 2 From the first target text information displayed in the lower part, it can be seen that the expanded first target text information is more natural, conforms to the language habits of the actual commentator in the actual scene, and improves the authenticity of the video.

[0053] In step 102, the first target text information in each sub-image is converted into speech to obtain an initial audio file corresponding to each sub-image, and the initial audio files corresponding to each sub-image are synthesized to obtain a target audio file for the static PPT.

[0054] Among them, a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT is constructed.

[0055] The corresponding relationship between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT means that the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT are the same. Figure 2 Taking one of the sub-images of the static PPT shown as an example, the sub-image is continuously displayed during the audio time until the audio time for the sub-image expires. The audio time of the initial audio file at least includes: the start time, end time, play duration and other parameters of the initial audio file.

[0056] Through this embodiment, a correspondence between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT is constructed, so as to avoid the situation where the audio of a sub-image is not played completely before switching to the next sub-image; or the audio of a sub-image is played completely but still not switched to the next sub-image, thereby avoiding the problem of transition jamming during video playback.

[0057] Exemplarily, the first target text information in each sub-image obtained in step 101 can be converted into speech, thereby obtaining an initial audio file corresponding to each sub-image. For example, the first target text information in each sub-image can be converted into audio using a speech-to-text conversion model (TTS); or the Whisper Speech method can be used to convert text into speech to form audio.

[0058] In the process of converting the first target text information in each sub-image into speech, the first target text information in each sub-image can be converted into speech based on the pre-configured timbre to obtain an initial audio file corresponding to each sub-image. Generally, the pre-configured timbre is consistent with the selected target digital human image. For example, if the selected target digital human image is a man, the corresponding configured timbre is a male voice; or if the selected target digital human image is a middle-aged man, the corresponding configured timbre is a medium-low male voice.

[0059] In step 102, the initial audio files corresponding to the sub-images are synthesized to obtain a target audio file for the static PPT; wherein a preset pause duration is added between the initial audio files in the target audio file.

[0060] Exemplarily, after the initial audio files corresponding to the sub-images are obtained based on step 102, the initial audio files corresponding to the sub-images may be synthesized to obtain a target audio file for the static PPT.

[0061] The process of synthesizing the target audio file is described below in conjunction with a specific embodiment:

[0062] Embodiment 1: In an optional embodiment of the present disclosure, a preset pause duration is added after the initial audio file corresponding to each sub-image; the initial audio file corresponding to each sub-image and the preset pause duration are synthesized to obtain a target audio file for a static PPT.

[0063] Exemplarily, after obtaining the initial audio files corresponding to each sub-image, a preset pause duration (for example, 1 second) can be added to the end of each initial audio file, so that the initial audio files corresponding to each sub-image and the preset pause duration can be synthesized to obtain the target audio file for the static PPT.

[0064] For example, assuming that the static PPT contains 3 PPTs, the 3 initial audio files corresponding to the 3 PPTs obtained based on the above step 102 are: input1.mp3, input2.mp3, input3.mp3, then the implementation logic of the entire target audio file obtained after integration is as follows:

[0065] ffmpeg-itsoffset 1-iinput1.mp3-itsoffset 1-iinput2.mp3-itsoffset 1-iinput3.mp3-filter_complex"amix=inputs=3:duration=first"output.mp3.

[0066] in:

[0067] -itsoffset 1: indicates that input1.mp3 is played with a delay of 1 second.

[0068] -filter_complex "amix=inputs=3:duration=first": represents mixing three input audio files and using the duration of the shortest file (ie duration=first) to determine the length of the final output.

[0069] output.mp3: represents the output target audio file.

[0070] It should be explained that, finally, make sure that the paths of input1.mp3, input2.mp3 and input3.mp3 are correct. After running the command, the output.mp3 file will contain the audio mixing result with the preset pause duration.

[0071] It can be seen from the above embodiment that a 1-second delay (i.e., a 1-second pause) is added after the three initial audio files corresponding to the three PPTs, so as to mix the three input audio files and obtain the output target audio file.

[0072] In addition to the above embodiments, in another optional embodiment of the present disclosure, the first target text information in each sub-image is integrated into the same text to obtain integrated second target text information; in the second target text information, a preset pause duration is added after the end text mark of the first target text information of each sub-image; the second target text information and the preset pause duration are converted into voice to obtain a target audio file for the static PPT.

[0073] Exemplarily, the first target text information in all sub-images in the PPT document can be integrated into the same text, and a pause mark can be added at the end of each PPT text content. Finally, all text and pauses are converted into audio together to obtain the target audio file.

[0074] The following will be combined Figure 3 An exemplary description is given. Figure 3 A schematic diagram of a synthetic target audio file provided in one embodiment of the present application, referring to Figure 3 The following are two PPT contents. The target text information of the first PPT is "Hello everyone. Today I will introduce the work results and future work plans for you." The target text information of the second PPT is "First, let's take a look at our catalog. The first part is the work overview, the second part is the completion status, the third part is the results display, and the last part is the work plan." Then add a 1-second pause mark at the end of the target text information of the first PPT. Finally, convert all text and pauses into audio together to get the target audio file.

[0075] Through the above-mentioned embodiments, it is convenient to establish a corresponding relationship between the audio time of each initial audio file and the display time of each corresponding sub-image in the static PPT, so as to synchronize the PPT content in the audio and video background from a time perspective, thereby avoiding the transition jamming problem in the related technical solutions and further improving the smoothness of video playback.

[0076] In step 103, a PPT transition animation with a preset pause length is added between each sub-image to obtain an initial PPT video for a static PPT.

[0077] In order to make the transition between different PPT pages more natural, a PPT transition animation can be added between each sub-image, and the duration of the PPT transition animation is the same as the preset pause duration constructed in the above step 102, thereby obtaining an initial PPT video for a static PPT.

[0078] Exemplarily, the PPT playback time is made to correspond to the time of each initial audio file, and a PPT transition animation is added to form a PPT video. That is, the initial audio files corresponding to each sub-image obtained in step 102 are recorded, and the audio time of the initial audio files corresponding to each sub-image is recorded, and the recorded audio time is mapped to the rehearsal time of each PPT to synthesize the initial PPT video for the static PPT. This process can strictly correspond the time when the virtual digital person tells the PPT content and the time when the PPT turns to the next page of PPT content, and the transition animation can be set to 1s (preset pause length).

[0079] The following will be combined Figure 4 The above-mentioned embodiments are described as examples. Figure 4 A schematic diagram of a PPT switching setting page provided in one embodiment of the present application, referring to Figure 4 As shown, set parameters such as the automatic switching speed and switching method of a single PPT page and PPT transition animation.

[0080] like Figure 4 As shown, after adding the transition animation of PPT, the speed is the time setting of PPT transition animation (i.e., the preset pause time), and the time of automatic film change is the display time that can be set in each PPT page of PPT, which needs to correspond to the audio time of the initial audio file of each sub-image, so as to ensure the consistency of the time of the converted PPT video and the time of the virtual digital human narrating the content. After the above setting operation is completed, a complete initial PPT video of PPT can be obtained.

[0081] In step 104, a virtual digital human is added to the initial PPT video, and the virtual digital human is controlled to explain the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video.

[0082] Exemplarily, a virtual digital human may be added to the initial PPT video, thereby driving the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file, so that the virtual digital human's lip shape corresponds to the narrated text information, thereby realizing the target PPT video in which the virtual digital human narrates the PPT content.

[0083] For virtual digital people, pre-selection can be performed, which will be described below in conjunction with specific embodiments:

[0084] In an optional embodiment of the present disclosure, a target digital human image is determined from candidate digital human images; and a virtual digital human is constructed based on pre-configured digital human parameters and the target digital human image.

[0085] The digital human parameters at least include: one or more of timbre, posture, expression, and body movements. The candidate digital human images are multiple digital human images that can be selected by users. The digital human image refers to the facial features of the digital human. The digital human image is mainly a 2D image type, thereby reducing the consumption of storage resources.

[0086] Exemplarily, the construction and selection of virtual digital humans can rely on a digital human technology platform, that is, the digital human technology platform contains multiple candidate digital human images, and provides selectable digital human parameters for each candidate digital human image. Since the digital human image is mainly a 2D image type, it is composed of a specific posture, a specific body movement and a specific expression. For example, the posture is sitting or standing, the body movement is such as raising the hand to indicate, nodding, etc., and the expression is smiling, serious, etc. During the video production process, the video resolution can be selected; the video is recorded in combination with standing and sitting postures, expressions such as smiling and surprised, and hand movements such as raising the hand to indicate.

[0087] The following will be combined Figure 5 To explain, Figure 5 A schematic diagram of a candidate digital human image provided in one embodiment of the present application; Figure 5 The figure shows three optional digital human images, all of which are standing postures, but the hand movements are different, and the expressions are also different. For example, the first virtual digital human has a smiling expression, while the second virtual digital human has no expression. That is, in the digital human template system, parameters such as digital human image movements and expressions can be selected to edit the image of the virtual digital human. It is understandable that if the user directly uses the digital human image mode, the virtual digital human will narrate the PPT content according to the default parameters. The audio files generated by the above embodiments can be formed by selecting different timbres according to the virtual digital human image and combining female voice, male voice, etc. to form different initial audio files.

[0088] It needs to be explained that Figure 5 The virtual digital human shown may be an anime cartoon character, a virtual human with the same image as an actual real person, or a virtual human formed by combining the two, etc. The embodiments of the present disclosure do not impose any special restrictions on this.

[0089] Furthermore, after selecting the virtual digital human to be added, the virtual digital human can be controlled to explain the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video.

[0090] In an optional embodiment of the present disclosure, audio features are extracted from a target audio file; a three-dimensional facial mesh is generated for a virtual digital human, and facial feature marker points based on a two-dimensional facial plane of the virtual digital human are projected; facial feature points in the three-dimensional facial mesh are controlled to be offset based on the audio features and a diffusion model to interpret the first target text information in an initial PPT video, thereby obtaining a target PPT video.

[0091] For example, the virtual digital person's mouth shape can be made to correspond to the text by using audio drive, so as to realize an animated video of the digital person telling the PPT content. It can use methods such as AniPortrait to make the virtual digital person's mouth shape correspond to the audio, so as to drive the virtual digital person to speak based on the target audio file.

[0092] Specifically, audio features are extracted first, and the digital human image is generated into a 3D facial mesh and posture. Then, 2D markers are used to form projections, and finally, the diffusion model is used for animation generation. In order to improve the accuracy of lip movements, AniPortrait introduces the PoseGuider module, which uses the multi-scale strategy of ControlNet to integrate marker features of different scales into different layers of the network. In addition, the markers of the reference image are included as additional inputs to enhance the accuracy of the generated animation through the cross-attention mechanism.

[0093] Figure 6 A schematic diagram of a video file of a static PPT narrated by a virtual digital human driven by audio provided in one embodiment of the present application; Figure 6 As shown, Figure 5 As an example, combining the first virtual digital human image with the target audio file can produce Figure 6 Video file shown.

[0094] Furthermore, use the PPT video as the background and place the above Figure 6 The virtual digital human narration animation shown in the figure can be synthesized as follows Figure 7 The target PPT video shown.

[0095] Reference Figure 7 As shown, the above embodiment obtains a smooth PPT video (i.e., the initial PPT video), and then obtains Figure 5 The animation of the virtual digital person telling the PPT content can be Figure 7As shown, the initial PPT video is used as the background and synthesized with the virtual digital human narration animation. The virtual digital human can be placed at the lower left or lower right position of the initial PPT video background, thereby forming a target PPT video of the virtual digital human narrating the PPT content. In this process, it is necessary to pay attention to the correspondence between the PPT video time and the digital human narration time.

[0096] Through the above embodiments, artificial intelligence methods such as using pre-trained models can make text-to-audio conversion and digital human narration of text simple and easy to operate.

[0097] In an optional embodiment of the present application, before the above step 101, extracting the first target text information in each sub-image from the sequence image set corresponding to the static PPT, the method further includes performing HDR (High Dynamic Range) image reconstruction on each sub-image to improve the image quality of each sub-image, and the reconstruction process at least includes:

[0098] Acquire each sub-image, and perform feature extraction on the sub-image to obtain an attention information feature map for the sub-image;

[0099] Inputting the attention information feature map into a pre-trained image reconstruction model to obtain a reconstructed high dynamic range HDR image; wherein the image reconstruction model at least includes a local enhancement module and a spatial feature transformation module, the local enhancement module is used to extract the corresponding local spatial information from the attention information feature map, and the spatial feature transformation module is used to perform a spatial affine transformation on the local spatial information to obtain an HDR image corresponding to each sub-image;

[0100] The first target text information in each corresponding sub-image is extracted based on each HDR image.

[0101] In an optional embodiment of the present application, the extracting features of the sub-image to obtain an attention information feature map for the single-frame sub-image includes:

[0102] Input the sub-image into a pre-trained Transformer model to extract multiple deep local feature representations from the sub-image based on the Transformer model, determine the relative position of each of the deep local feature representations, and generate a relative position code of each of the deep local feature representations;

[0103] The multiple deep local feature representations and the relative position encoding of each of the deep local feature representations are input into a pre-trained SwinIR model for feature extraction to obtain an attention information feature map for the sub-image.

[0104] In an optional embodiment of the present application, the inputting the attention information feature map into a pre-trained image reconstruction model to obtain a reconstructed high dynamic range HDR image includes:

[0105] Performing dimensionality reduction processing on the attention information feature map to convert the image size of the attention information feature map to be the same as the size of the sub-image;

[0106] The attention information feature map after dimensionality reduction processing is input into the image reconstruction model to obtain a reconstructed HDR image.

[0107] In an optional embodiment of the present application, the local enhancement module includes at least a residual block structure, and the residual block structure includes a dense connection layer, a local feature fusion layer and a local residual learning layer. The dense connection layer, the local feature fusion layer and the local residual learning layer are respectively used to extract corresponding local spatial information from the attention information feature map.

[0108] In an optional embodiment of the present application, the local enhancement module also includes a UNet network structure, and the UNet network structure is before the residual block structure, so as to reconstruct the image based on the UNet network structure through the attention information feature map.

[0109] In an optional embodiment of the present application, the method further includes:

[0110] Performing spatial affine transformation on the local spatial information according to the spatial feature transformation module in the image reconstruction model to obtain image features of the HDR image;

[0111] Performing feature fusion on the image features of the HDR image to obtain an initial HDR image;

[0112] Perform tone mapping on the initial HDR image to obtain a reconstructed HDR image.

[0113] In an optional embodiment of the present application, the performing tone mapping on the initial HDR image to obtain a reconstructed HDR image includes:

[0114] Based on the Gamma tone mapping function or the μ-Law tone mapping function, tone mapping is performed on the initial HDR image to obtain a reconstructed HDR image.

[0115] In an optional embodiment of the present application, before extracting the first target text information in each sub-image from the sequence image set of the static PPT in the above step 101, the method further includes reviewing the graphic content of each sub-image, and the review process includes:

[0116] Based on the pre-configured unstructured data processing module, a multimodal large model is used to perform semantic analysis on the unstructured data such as news pictures, audio, and video in each sub-image, generate a semantic association graph, extract the core context and information relevance in the text and image through in-depth analysis, and construct a preliminary review data set;

[0117] Based on the pre-configured multimodal data integration module, the graphic data, audio data and video data in each sub-image are integrated in a unified context, converted into a graphic-text association vector space, and transmitted to the content analysis module for in-depth analysis of content context and emotion;

[0118] The content analysis module is used to preliminarily determine the degree of matching between the text and the image content based on the semantic association graph generated by the unstructured data processing module, generate a context factor Qycs, and analyze the potential sensitive content in each modal data to generate a sensitivity factor Gmyz; then, based on the context factor Qycs and the sensitivity factor Gmyz, perform hierarchical analysis on the potential emotional factors, social sensitive information and potential misleading factors in the image and text content to generate a multimodal association factor Mmgyz;

[0119] The content anomaly quantification module calculates the audit weight value Hqwz based on the situational factor Qycs, the sensitivity factor Gmyz and the multimodal association factor Mmgyz; then obtains the system audit risk factor Rxz by calculation and associates it with the audit weight value Hqwz, thereby generating the final content anomaly index Cxzs;

[0120] Among them, the calculation formula of the audit weight value Hqwz is:

[0121] Hqwz=b1×Qyca+b2×Gmyz+b3×Mmgyz+B;

[0122] Among them, b1, b2 and b3 represent the preset weight coefficients of the situational factor Qycs, the sensitive factor Gmyz and the multimodal association factor Mmgyz respectively, and B represents the audit correction coefficient;

[0123] The intelligent control module is used to set the compliance threshold T and the audit risk threshold R, and compare the audit weight value Hqwz with the compliance threshold T to determine whether the current content meets the audit standards, and compare the audit risk threshold R with the content anomaly index Cxzs to obtain the final audit result, and selectively trigger manual review or automatic release control plans based on the audit results.

[0124] In an optional embodiment of the present application, the unstructured data processing module is first used to receive the unstructured data of the image and text in each sub-image and perform format standardization processing, including image resolution adjustment and text language normalization processing;

[0125] Then, the pre-trained network of the multimodal large model is called to extract multi-level features, including visual features, semantic features and emotional features, from the standardized unstructured data in each sub-image, and the extracted multi-level features are normalized to generate a unimodal feature matrix; based on the unimodal feature matrix, the multi-level features of the image and text in each sub-image are mapped in the multimodal association vector space, and a cross-modal semantic association graph is generated through the semantic embedding mechanism of the multimodal large model;

[0126] The cross-modal semantic association graph is used to conduct a detailed analysis of the deep-level association between the text and each sub-image, extract the core contextual elements in the text and image, and simultaneously analyze the emotional state and expression trend therein, so as to identify the potential sensitive information, misleading information and social influencing factors in the text and image content in real time, and form a multi-modal association semantic index;

[0127] Finally, a preliminary review dataset is generated.

[0128] In an optional embodiment of the present application, the method further includes: calling the multimodal data integration module to receive the preliminary review data set generated by the unstructured data processing module, and uniformly aligning the timing and content of the graphic data, audio data and video data therein through a frame synchronization algorithm;

[0129] A multimodal feature mapping algorithm is used to fuse the context, emotion and semantic features of the image and text into a multimodal feature vector, and a correlation matrix is ​​generated based on the correlation between the modalities to perform hierarchical mapping of information density, context relevance and semantic consistency.

[0130] Based on the association matrix, the image-text association vector space is constructed, and the multimodal data is quantified in a unified semantic space to generate an integrated semantic association representation vector. At the same time, the semantic deviation caused by modality differences is corrected through the context adjustment algorithm.

[0131] The integrated image-text association vector space is transmitted to the content analysis module for analysis.

[0132] In an optional embodiment of the present application, by analyzing the semantic similarity and contextual consistency between the image and the text, the semantic deviation related data and the content matching related data are identified; the semantic similarity Ycy, contextual consistency Sjy, contextual correlation deviation Qjl and image-text emotional consistency Twq in the semantic deviation related data and the content matching related data are extracted and dimensionlessly processed, and the contextual factor Qycs is calculated by the following formula:

[0133]

[0134] Among them, semantic similarity Ycy is used to measure the similarity between the text content and the content expressed by the image at the semantic level, and the text vector representation is obtained by inputting the text into the pre-trained multimodal model; context consistency Sjy mainly measures the consistency of multidimensional information such as time, place, person or event between the text context and the image context; situational association deviation Qjl is used to quantify the potential situational mismatch between text and image, and it is obtained by aligning the text and image, identifying the inconsistent parts of the "core scene or main object", and counting the inconsistent categories or numbers; picture-text emotional consistency Twq is to measure the degree of consistency between the emotions conveyed by the text and the emotions conveyed by the image, and it is obtained by calculating the similarity between the text emotional distribution and the image emotional distribution;

[0135] The preset situation threshold Q is compared with the situation factor Qycs to evaluate the situational relevance of text and images in the audit scenario. The specific comparison and evaluation contents are as follows:

[0136] If the context factor Qycs ≥ the context threshold Q, it indicates that the contexts of the text and the image match, and the semantics, emotions, and contexts of the two are consistent. The review is passed and marked as "normal context association";

[0137] If the context factor Qycs < context threshold Q, it indicates that the context between the text and the image does not match, and the semantics, emotions and context of the two are inconsistent. The review will fail and further analysis of the specific reasons for the deviation will be conducted, including potential sensitive content.

[0138] In an optional embodiment of the present application, the multi-layer sensitivity analysis unit is used to further perform in-depth analysis on the potential sensitive content of the graphic data, including analyzing the emotion-related data, social sensitivity-related data and potential misleading-related data in the text and image, and constructing a sensitive content data set after summarizing and dimensionlessly processing the emotion-related data, social sensitivity-related data and potential misleading-related data in the analyzed text and image; extracting the sensitive content data set and generating the sensitivity factor Gmyz through the following formula:

[0139]

[0140] Where Qmq represents the emotion intensity coefficient in the sensitive content data set, Mgc represents the sensitive word index in the sensitive content data set, and Xwg represents the possibility of information misleading in the sensitive content data set.

[0141] In an optional embodiment of the present application, the correlation factor generation unit is used to quantify the correlation degree between emotion, context and sensitive content based on the context factor Qycs and the sensitivity factor Gmyz by fusing multi-level information in the image-text correlation vector space, and generate a multimodal correlation factor Mmgyz by calculating through the following formula;

[0142]

[0143] In an optional embodiment of the present application, the content anomaly quantification module includes an audit calculation unit and an anomaly index acquisition unit;

[0144] The content evaluation unit is used to construct a system audit risk factor Rxz, obtain the semantic deviation degree Ycp and the content consistency index Nry by extracting semantic deviation related data and content matching related data, and obtain the high sensitivity trigger rate Gmg by extracting the sensitive content data set, and calculate the system audit risk factor Rxz in combination with the following formula:

[0145]

[0146] In an optional embodiment of the present application, the abnormality index obtaining unit is used to calculate and obtain the final content abnormality index Cxzs, and its specific calculation formula is as follows:

[0147]

[0148] In an optional embodiment of the present application, the intelligent control module compares and evaluates the audit weight value Hqwz with the compliance threshold T, and compares and evaluates the audit risk threshold R with the content anomaly index Cxzs, and specifically generates the following evaluation content:

[0149] Compliance standards comparison:

[0150] If the audit weight value Hqwz ≥ the compliance threshold T, it means that the content meets the audit standards and the system marks it as "compliant"

[0151] If the audit weight value Hqwz < the compliance threshold T, it means that the content does not meet the audit standards and is insufficiently compliant, requiring further review or adjustment.

[0152] Abnormal risk assessment:

[0153] If the content anomaly index Cxzs ≥ the audit risk threshold R, it indicates that the content has an abnormal risk, and the system generates an "abnormal content" mark, triggering the manual review process;

[0154] If the content anomaly index Cxzs < the audit risk threshold R, it means that there is no abnormal risk in the content, and the system generates a "normal content" mark, and then enters the automatic publishing process;

[0155] When the evaluation results of the audit weight value Hqwz and the content anomaly index Cxzs are marked as "compliant" and "normal content" at the same time, the content will be automatically marked as "passed" and published directly;

[0156] When the evaluation result of the audit weight value Hqwz and the content anomaly index Cxzs is that the content does not meet the audit standards or has abnormal risks, the content will be automatically marked as "pending review" and the selective review process will be triggered.

[0157] It is used to set the compliance threshold T and the audit risk threshold R, and to determine whether the current content meets the audit standards, obtain the final audit results, and selectively trigger manual review or automatic release control plans based on the audit results.

[0158] It should be understood that, although the various steps in the flow chart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the figure may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0159] In order to realize the above-mentioned method of making a video of a digital human automatically explaining a static PPT, please refer to Figure 8 An embodiment of the present application provides a video production device 800 in which a digital human automatically explains a static PPT. The video production device 800 in which a digital human automatically explains a static PPT may include: a text extraction module 801, an audio synthesis module 802, an animation adding module 803 and a control module 804.

[0160] Among them, the text extraction module 801 is used to extract the first target text information in each sub-image from the sequence image set of the static PPT; the audio synthesis module 802 is used to convert the first target text information in each sub-image into voice, obtain the initial audio file corresponding to each sub-image, and synthesize the initial audio files corresponding to each sub-image to obtain the target audio file for the static PPT; wherein, a preset pause duration is added between each initial audio file in the target audio file, and a corresponding relationship between the audio time of each initial audio file and the display time of each sub-image corresponding to the static PPT is constructed; the animation adding module 803 is used to add a PPT transition animation with a preset pause duration between each sub-image to obtain the initial PPT video for the static PPT; the control module 804 is used to add a virtual digital human to the initial PPT video, and control the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video.

[0161] In an optional embodiment of the present disclosure, the text extraction module 801 is specifically used to obtain a static PPT; store the static PPT in an image format to obtain a sequence image set corresponding to the static PPT; or, take a screenshot of each PPT page in the static PPT to obtain a sequence image set corresponding to the static PPT; or, process the static PPT based on a preset conversion model or class file to obtain a sequence image set corresponding to the static PPT; and extract the first target text information in each sub-image from the sequence image set of the static PPT.

[0162] In an optional embodiment of the present disclosure, the text extraction module 801 is specifically used to extract the initial text information in each sub-image from the sequence image set corresponding to the static PPT; based on the image-text multimodal understanding model, the initial text information is expanded to obtain the first target text information in each sub-image.

[0163] In an optional embodiment of the present disclosure, the audio synthesis module 802 is used to add a preset pause duration after the initial audio file corresponding to each sub-image; the initial audio file corresponding to each sub-image and the preset pause duration are synthesized to obtain a target audio file for the static PPT.

[0164] In an optional embodiment of the present disclosure, the audio synthesis module 802 can also be used to integrate the first target text information in each sub-image into the same text to obtain integrated second target text information; in the second target text information, add a preset pause time after the end text mark of the first target text information of each sub-image; convert the second target text information and the preset pause time into speech to obtain a target audio file for the static PPT.

[0165] In an optional embodiment of the present disclosure, the device may also include an image determination module and a digital human construction module, the image determination module is used to determine the target digital human image from the candidate digital human images; the digital human construction module is used to construct a virtual digital human according to pre-configured digital human parameters and the target digital human image; the digital human parameters include at least: one or more of: timbre, posture, expression, and body movement.

[0166] In an optional embodiment of the present disclosure, a control module is used to extract audio features from a target audio file; generate a three-dimensional facial mesh for a virtual digital human, and project facial feature marker points based on a two-dimensional facial plane of the virtual digital human; and control the facial feature points in the three-dimensional facial mesh to offset based on the audio features and a diffusion model to explain the first target text information in an initial PPT video to obtain a target PPT video.

[0167] For the specific definition of the above-mentioned video production device for automatically explaining a static PPT with a digital human, please refer to the definition of the video production method for automatically explaining a static PPT with a digital human, which will not be repeated here. Each module in the above-mentioned video production device for automatically explaining a static PPT with a digital human can be implemented in whole or in part by software, hardware, and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0168] In one embodiment, a computer device is provided, the internal structure diagram of which can be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video production method of a digital person automatically explaining a static PPT as described above is implemented. It includes: a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, any step in the video production method of a digital person automatically explaining a static PPT as described above is implemented.

[0169] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, any step in the above-mentioned method for producing a video of a digital human automatically explaining a static PPT can be implemented.

[0170] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0171] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0172] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0173] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0174] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0175] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for producing a video of a digital human automatically explaining a static PPT, characterized in that: include: Extracting the first target text information in each sub-image from the sequence image set of the static PPT; The first target text information in each of the sub-images is converted into speech to obtain an initial audio file corresponding to each sub-image, and the initial audio files corresponding to each of the sub-images are synthesized to obtain a target audio file for the static PPT; wherein a preset pause length is added between each of the initial audio files in the target audio file, and a corresponding relationship between the audio time of each of the initial audio files and the display time of each of the sub-images in the static PPT is constructed; Adding a PPT transition animation with a preset pause time between each of the sub-images to obtain an initial PPT video for the static PPT; A virtual digital human is added to the initial PPT video, and based on the target audio file, the virtual digital human is controlled to explain the first target text information in the initial PPT video to obtain a target PPT video.

2. The method according to claim 1, characterized in that The step of extracting the first target text information in each sub-image from the sequence image set of the static PPT includes: Obtain the static PPT; The static PPT is stored in an image format to obtain a sequence image set corresponding to the static PPT; or, each PPT page in the static PPT is screenshoted to obtain a sequence image set corresponding to the static PPT; or, the static PPT is processed based on a preset conversion model or class file to obtain a sequence image set corresponding to the static PPT; The first target text information in each sub-image is extracted from the sequence image set of the static PPT.

3. The method according to claim 1 or 2, characterized in that: Extracting the first target text information in each sub-image from a sequence image set of a static PPT includes: Extracting initial text information in each sub-image from a set of sequential images corresponding to the static PPT; The initial text information is expanded based on the image-text multimodal understanding model to obtain the first target text information in each sub-image.

4. The method according to claim 1, characterized in that: The initial audio files corresponding to the sub-images are synthesized to obtain a target audio file for the static PPT, including: adding the preset pause duration after the initial audio files corresponding to the sub-images; The initial audio files corresponding to the sub-images and the preset pause duration are synthesized to obtain a target audio file for the static PPT.

5. The method according to claim 1, characterized in that The method further comprises: Integrate the first target text information in each of the sub-images into the same text to obtain integrated second target text information; In the second target text information, the preset pause time is added after the end text mark of the first target text information of each sub-image; The second target text information and the preset pause duration are converted into voice to obtain the target audio file for the static PPT.

6. The method according to claim 1, characterized in that The method further comprises: Determine a target digital human image from candidate digital human images; The virtual digital human is constructed based on pre-configured digital human parameters and the target digital human image; the digital human parameters at least include: one or more of timbre, posture, expression, and body movements.

7. The method according to claim 1, characterized in that The step of controlling the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file to obtain the target PPT video includes: Extracting audio features from the target audio file; Generating a three-dimensional facial mesh for the virtual digital human, and projecting facial feature marker points based on the two-dimensional facial plane of the virtual digital human; Based on the audio features and the diffusion model, the facial feature points in the three-dimensional facial mesh are controlled to be offset so as to explain the first target text information in the initial PPT video and obtain a target PPT video.

8. A video production device for a digital human to automatically explain a static PPT, characterized in that: include: A text extraction module, used to extract the first target text information in each sub-image from the sequence image set of the static PPT; An audio synthesis module, used to convert the first target text information in each of the sub-images into speech, obtain an initial audio file corresponding to each of the sub-images, and synthesize the initial audio files corresponding to each of the sub-images to obtain a target audio file for the static PPT; wherein a preset pause length is added between each of the initial audio files in the target audio file, and a corresponding relationship between the audio time of each of the initial audio files and the display time of each of the sub-images in the static PPT is established; An animation adding module, used for adding the PPT transition animation with the preset pause time between each of the sub-images to obtain an initial PPT video for the static PPT; The control module is used to add a virtual digital human to the initial PPT video, and control the virtual digital human to explain the first target text information in the initial PPT video based on the target audio file to obtain a target PPT video.

9. A computer device comprising: It comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the method for producing a video of a digital human automatically explaining a static PPT as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for producing a video of a digital human automatically explaining a static PPT according to any one of claims 1 to 7 are implemented.

Images