Digital human video generation method and device based on AIGC technology

By using a digital human video generation method based on AIGC technology, the challenges of personnel training and video generation in corporate presentations have been solved. This method enables automated and diversified digital human presentation video generation, improving presentation efficiency and quality while reducing costs.

CN118945440APending Publication Date: 2024-11-12欧冶云商股份有限公司
View PDF 0 Cites 4 Cited by

Patent Information

Application Number
CN202411183734.8
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

Existing technologies for corporate presentations suffer from drawbacks such as significant time commitment for training presenters, large variations in quality, and heavy pressure on image management. Furthermore, digital human video generation solutions struggle to effectively integrate with conventional documents and ensure image consistency.

Method used

A digital human video generation method based on AIGC technology is adopted. By unifying material processing through a configuration table, a digital human image library is built, and AIGC image technology is used for automated generation. Combined with speech synthesis and micro-editing, a variety of presentation videos are generated.

Benefits of technology

It enables the automated generation of digital human presentation videos, ensuring video diversity and image consistency, reducing user operation costs, improving presentation efficiency and quality, saving labor costs, and providing micro-editing functions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945440A_ABST
    Figure CN118945440A_ABST
Patent Text Reader

Abstract

The invention relates to a digital human video generation method based on the AIGC technology, and the method comprises the steps: obtaining a human picture, a configuration table, a material file and a background material, and the configuration table comprises the sub-mirror configuration, the video configuration and the voice configuration; the figure pictures are processed through the AIGC image technology, and a digital figure picture library is constructed; converting the material file into a picture file; synthesizing broadcast voice and a subtitle file according to the sub-mirror configuration and the voice configuration; extracting a digital human picture from the digital human picture library according to the sub-mirror configuration, extracting voice features from the broadcast voice by using a digital human driving technology, driving the digital human picture, and generating a digital human broadcast video matched with the broadcast voice; and integrating the materials, performing micro editing according to the configuration table, and outputting a final digital human video. Compared with the prior art, by using the AIGC technology and the template generation process of the digital human preaching video, the automation of the generation process of the digital human preaching video is realized, and meanwhile, the diversity of the video is ensured.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a method and device for generating digital human videos based on AIGC technology. Background Art

[0002] In business operations, external product presentations and internal training exchanges are mostly conducted manually, which requires frequent training of presentation staff, easily taking up working time, and the quality of presentations by different people varies greatly, which is not conducive to the company's image display. At the same time, presentation staff also need to repeatedly face the challenge of content updates and the pressure of image management. In order to improve the efficiency and quality of presentations, many companies have adopted digital people to replace manual product presentations. For example, Chinese patent application CN115515002A discloses an intelligent MOOC generation method device and storage medium based on virtual digital human, which uses three-dimensional digital human technology to generate MOOC videos. Chinese patent application CN117336568A discloses a digital human video generation method device electronic device and storage medium, which uses template thinking to generate digital human videos. The former often requires pre-training of a three-dimensional image model of the digital human, and its 3D modeling, 3D data acquisition, and model training are difficult and demanding. Although the latter focuses on video template thinking to generate videos, which can reduce editing requirements, the material docking and presentation are mainly based on pictures or videos, and ignores the docking and fusion with conventional files such as PPT, word and pdf. At the same time, the generation of the digital human image in this scheme does not perform separate masking or training processing on the face and clothing, and it is impossible to specify the digital human clothing, which also easily makes the generated digital human facial image have a large gap with the original character image. Summary of the invention

[0003] The purpose of the present invention is to provide a method and device for generating digital human videos based on AIGC technology. By utilizing AIGC technology and the generation process of templated digital human presentation videos, the generation process of digital human presentation videos is automated while also ensuring the diversity of videos.

[0004] The purpose of the present invention can be achieved by the following technical solutions:

[0005] A method for generating digital human video based on AIGC technology, the method comprising:

[0006] Obtain character pictures, configuration tables, material files and background materials. The configuration table includes storyboard configuration, video configuration and voice configuration;

[0007] Use AIGC image technology to process human images and build a digital human image library;

[0008] Preprocessing the material file and converting it into an image file, wherein each image in the image file corresponds to a storyboard in the storyboard configuration;

[0009] According to the storyboard configuration and voice configuration, synthesize the broadcast voice and subtitle files according to the storyboard;

[0010] Extract digital human images from the digital human image library according to the storyboard configuration, extract voice features from the broadcast voice using digital human driving technology and drive the digital human images to generate a digital human broadcast video that matches the broadcast voice;

[0011] Combine the background material and the picture files of each storyboard, the broadcast voice, the subtitle files and the digital human broadcast video, perform micro-editing according to the configuration table, and output the final digital human video.

[0012] Furthermore, the split mirror configuration includes:

[0013] INDEX, storyboard number;

[0014] TEXT: the text that needs to be broadcast or displayed in the storyboard;

[0015] TYPE, whether the storyboard needs to generate text animation video;

[0016] BG_FILE, storyboard background file, the formats include pictures and videos;

[0017] DH_POSITION, the position of the digital human in the storyboard video, including center, right, left, lower left, and lower right;

[0018] DH_SHOW, the presentation style of the storyboard digital human;

[0019] DH_FILE, digital human image extracted from the digital human image library;

[0020] TRANSITION, the transition animation between the current storyboard and the next storyboard.

[0021] Furthermore, the video configuration is used to set the parameters used in the final digital human video output process, including the height and width of the video, whether to add subtitles and background sound files, and the names of the added subtitle files and background sound files.

[0022] Furthermore, the voice configuration is used to set the parameters used in the speech synthesis process, including the pronunciation role name, speaking speed, volume and audio sampling rate.

[0023] Furthermore, the step of processing the character image using the AIGC technology includes:

[0024] Screening the person pictures, retaining the pictures with the same size, clear boundaries between the background and the person, and different actions of the person that meet the action display requirements in different scenes, wherein the action display requirements include the finger pointing to the left and the finger pointing to the right;

[0025] Name the filtered pictures with the action name;

[0026] Use AIGC image technology to segment the screened and named person images into different blocks, retain the clothing area, and generate clothing masks;

[0027] According to the presentation style of the digital human in the storyboard configuration, fill in the positive and negative prompts, use AIGC technology to combine the image and the clothing mask to redraw the image within the mask, and generate the image of the character after changing clothes;

[0028] Using AIGC image technology, the poses of the characters after changing clothes are guided to generate characters with different actions;

[0029] Filter the generated images and name them after the action;

[0030] Output the digital human image after the change of clothes and the corresponding original image.

[0031] Furthermore, the preprocessing includes:

[0032] If the source file is in PPT format, convert the source file into pictures page by page, and rename the pictures in order according to the storyboard numbers in the storyboard configuration;

[0033] If the material file is a picture, it is named in the order of display number, and the picture order is renamed according to the storyboard number in the storyboard configuration;

[0034] If the source file is in word format, convert the file to pdf format and then convert it into pictures page by page, and rename the pictures in order according to the storyboard numbers in the storyboard configuration;

[0035] If the source file is in PDF format, convert the file into images page by page, and rename the images in order according to the storyboard numbers in the storyboard configuration;

[0036] Organize the renamed pictures and output the picture files.

[0037] Furthermore, the step of synthesizing the broadcast voice and subtitle files includes:

[0038] Set the speaker's timbre, speaking speed and sampling rate according to the voice configuration;

[0039] According to the storyboard configuration, speech synthesis technology is used to synthesize the broadcast voice and subtitle files containing timestamps.

[0040] Furthermore, the micro-editing includes:

[0041] Obtain the storyboard type parameters in the storyboard configuration, combine the picture file, broadcast voice, subtitle file and digital human broadcast video, and generate each storyboard video;

[0042] According to the storyboard configuration, add the specified transition animation to the storyboard room;

[0043] Use video processing tools to connect storyboard videos in sequence;

[0044] According to the video configuration, add background sound to the video and synthesize the final digital human video.

[0045] Furthermore, the storyboard type parameters are TEXT and DH_SHOW in the storyboard configuration, wherein:

[0046] When TEXT is text, it means generating a 3s text animation video according to the text that needs to be broadcast or displayed in the storyboard configuration corresponding to the storyboard;

[0047] When TEXT is empty, it means that the storyboard does not need to generate a text animation video;

[0048] When DH_SHOW is with_bg, it means extracting the digital human broadcast video with a transparent background of the storyboard, and then combining the digital human broadcast video with a transparent background of the storyboard, the material picture and the background material to generate a storyboard video with the same duration as the corresponding broadcast voice;

[0049] When DH_SHOW is only, it means extracting the digital human broadcast video with a transparent background of the storyboard, and then combining the digital human broadcast video with a transparent background of the storyboard with the background material to generate a storyboard video with the same duration as the corresponding broadcast voice;

[0050] When DH_SHOW is empty, it means generating a storyboard video that only displays the background material and the storyboard image file.

[0051] A digital human video generation device based on AIGC technology, the device comprising:

[0052] The material receiving module receives character pictures, configuration tables, material files and background materials. The configuration table includes storyboard configuration, video configuration and voice configuration.

[0053] The material processing module uses AIGC image technology to process character images, build a digital human image library, pre-process the material files, and convert them into image files. Each image in the image file corresponds to a storyboard in the storyboard configuration;

[0054] The video generation module synthesizes the broadcast voice and subtitle files according to the storyboard configuration and voice configuration, extracts the digital human image from the digital human image library according to the storyboard configuration, extracts the voice features from the broadcast voice and drives the digital human image by using the voice synthesis technology, extracts the voice features from the broadcast voice, drives the digital human image, and generates the digital human broadcast video matching the broadcast voice;

[0055] The video editing module integrates background materials and picture files of each storyboard, broadcast voice, subtitle files and digital human broadcast video, and performs micro-editing according to the configuration table;

[0056] The video output module outputs the final digital human video and uploads it for storage.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] 1. The present invention adopts a unified configuration table to count video parameters, unifies the material processing ideas, and templates the generation process of digital human presentation videos. At the same time, AIGC image technology is used to establish a 2D digital human picture library. Only by inputting photos, the digital human can be automatically dressed up while ensuring that the image of the digital human before and after processing is consistent. This realizes the automation of the digital human presentation video generation process while also ensuring the diversity of the video;

[0059] 2. The present invention only requires users to provide photos. Compared with other video digital human solutions that require the provision of video materials, the present invention has fewer application restrictions and lower costs;

[0060] 3. The present invention uses AIGC technology to optimize real-life or cartoon pictures, which can beautify the character image and reduce the user's operating costs. Only a small number of photos need to be taken to build a digital human picture library with rich clothing styles;

[0061] 4. The present invention has a micro-editing function, focusing on the combination of digital people and materials, and effectively improving the production efficiency of the presentation video by designing page layout, adding transitions and opening and ending credits, etc.;

[0062] 5. The present invention uses speech synthesis technology to present the content of the materials. Compared with the traditional method of manual presentation of materials, it can save labor costs and improve the standardization of the presentation content. At the same time, by introducing digital humans, the affinity of AI presentation can also be enhanced;

[0063] 6. The digital human images generated by the present invention can be reused later, and the voice, video and subtitles can also be used for backup and information review. The technical paths adopted are all implemented at the software level, and the basic functions are highly reusable without the need for additional hardware support. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 is a flow chart of the method of the present invention;

[0065] Figure 2 It is a schematic diagram of the workflow of the present invention;

[0066] Figure 3 It is a schematic diagram of the video template screen of the present invention. DETAILED DESCRIPTION

[0067] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0068] Example 1

[0069] The present invention provides a method for generating digital human video based on AIGC technology, such as Figure 1 As shown, the method includes:

[0070] S1, obtaining character pictures, configuration tables, material files and background materials, the configuration table including storyboard configuration, video configuration and voice configuration;

[0071] S2, using AIGC image technology to process character images, build a digital human image library, pre-process the material files, and convert them into image files, each image in the image file corresponds to a storyboard in the storyboard configuration;

[0072] S3, synthesizing the broadcast voice and subtitle files according to the storyboard configuration and voice configuration, extracting the digital human image from the digital human image library according to the storyboard configuration, extracting the voice features from the broadcast voice and driving the digital human image using the digital human driving technology, and generating a digital human broadcast video matching the broadcast voice;

[0073] S4, integrates the background material and the picture files of each storyboard, the broadcast voice, the subtitle file and the digital human broadcast video, performs micro-editing according to the configuration table, and outputs the final digital human video;

[0074] In summary, the present invention is based on AIGC image technology and template function design, and realizes multimedia stream combination and generates digital human presentation videos through 2D digital human application scenarios.

[0075] When actually using video to generate work, the steps are as follows:

[0076] Before starting the video generation process, you need to organize the relevant character pictures in advance, and then use the AIGC dressing stream processing to build a digital human picture library.

[0077] The specific steps of AIGC dressing flow are as follows:

[0078] Screening the person pictures, retaining the pictures with the same size, clear boundaries between the background and the person, different actions of the person, and the overall adaptability to the action display requirements in different scenes, wherein the action display requirements include the finger pointing to the left and the finger pointing to the right;

[0079] Name the filtered pictures with the action name;

[0080] Use the Segment Anything plug-in of stable-diffusion to divide the filtered and named person images into different blocks, click to keep the clothing area, and generate a clothing mask;

[0081] Use the image to partially redraw the image, combine the image and the clothing mask to redraw the image within the mask, and generate the image of the character after changing clothes;

[0082] Use the controlnet plug-in to guide the posture of the characters after changing clothes and generate characters with different actions;

[0083] According to DH_SHOW in the storyboard configuration, fill in the positive and negative prompts, generate and filter pictures, and name the filtered pictures with actions;

[0084] Output the digital human image after the change of clothes and the corresponding original image.

[0085] After that, the user prepares the material and fills in the configuration table (the following material is in PPT format as an example). After uploading, the video automatic generation process can be started. Figure 2 The specific operation steps are as follows:

[0086] 1. The user prepares materials such as introduction text, presentation PPT, background image or video, and background music;

[0087] 2. The user fills in the configuration form, including the storyboard configuration, video configuration and voice configuration, and uploads the presentation PPT, background video or picture, background music, etc. prepared in step 1;

[0088] 3. After receiving the relevant materials (in PPT format as an example) and the configuration table, the system performs preprocessing operations, converts the PPT into pictures by page, and renames the pictures with the row numbers in the storyboard configuration. The specific logic is: if the storyboard type parameter in the "storyboard configuration" is not "only", it means that the material is to be displayed, and the system will name the PPT page pictures with the storyboard row numbers, otherwise skip the generation of the PPT pictures for the storyboard;

[0089] 4. After the PPT page is converted into a picture, the speech synthesis function is started to read the text to be broadcasted in the storyboard configuration line by line, and generate the corresponding speech and subtitle files containing timestamp information for each storyboard according to the parameters such as speaker, speech speed and sampling rate set in the speech configuration;

[0090] 5. After the speech synthesis is started, start the digital human video generation function, read the speech synthesized in step 4 in sequence, and then use the digital human driving module to extract the speech features, drive the digital human image set in the storyboard configuration, and generate the digital human broadcast video of each storyboard that matches the speech;

[0091] 6. After obtaining the corresponding announcement voice, subtitle files and digital human broadcast video for each storyboard, start the video template service. Read the relevant contents of the configuration table row by row to generate the initial video for each storyboard:

[0092] If the storyboard type parameter is "text", the text of the storyboard is read first, and then the audio and video development tool editly is called to generate a storyboard video with a resolution consistent with the specified video configuration and a duration of 3s. This type of storyboard video is often used in the opening and ending credits, and the text to be displayed will be displayed in the center and have certain animation effects;

[0093] If the storyboard type parameter is "only", the digital human broadcast video will be read, and then the video cutout algorithm will be called to extract the digital human video with a transparent background. Finally, the opencv tool will be used to combine the digital human broadcast video with the preset background material frame by frame to generate the corresponding storyboard initial video file;

[0094] If the storyboard type parameter is "with_bg", the corresponding PPT page image, background image or video, digital human broadcast video and storyboard voice file will be read. First, the digital human broadcast video with a transparent background is extracted using a cutout model based on a deep learning algorithm; then the digital human size is scaled according to the preset parameters, and the Figure 3 Instructions: According to the position parameters of the digital human display (left or right or center or lower right or lower left), the transparent digital human video is superimposed on the background image or video; finally, the material image is superimposed on the background image or video according to the preset position (left or right, not overlapping with the digital human), and the initial storyboard video file with the same length as the corresponding voice under the storyboard type is generated;

[0095] If the storyboard type parameter is empty, only the image material and background image or video of the PPT page will be displayed. The specific process is to first read the background image or video, then read the material image, and use opencv to overlay the material image on the background image or video frame by frame to generate a storyboard initial video with the same duration as the corresponding voice.

[0096] 7. After the synthesis of the initial videos of each storyboard is completed, read the initial storyboard videos, the corresponding voice and subtitle files in turn, and then use the ffmpeg tool to add the corresponding voice of the same length to each initial storyboard video, and add the subtitle file at the specified position of the video to generate the process video corresponding to each storyboard unit;

[0097] 8. After the process videos corresponding to each storyboard unit are generated, the ffmpeg tool will be used to connect the storyboard videos in series, and the specified transition animation will be added to the storyboard according to the "TRNSITION" column value of each storyboard unit in the "Storyboard Type";

[0098] 9. After the transition animation is added, connect the storyboard clips in sequence, and add background sound to the video according to the background sound mp3 file in the "Video Configuration", and finally synthesize the final digital human broadcast video;

[0099] 10. After generating the final digital human broadcast video, upload the video to the S3 storage bucket, and the user can log in to S3 to download the broadcast video (see Figure 1 solid black arrow).

[0100] The above storyboard configuration table is mainly used to set the relevant parameters of the storyboard video, such as the storyboard video type, digital human display method, background fusion method, and transition animation, etc. The specific diagram can be seen in Table 1.

[0101] Table 1 Storyboard configuration table

[0102]

[0103] In Table 1, the meaning of each field is as follows:

[0104] "INDEX" is the storyboard number;

[0105] The "TEXT" column is the text that is broadcast or displayed in each storyboard;

[0106] In the "TYPE" column, if the value is "text", it means that the corresponding "TEXT" value is used to generate a 3s text animation video; if it is empty, it means that the storyboard does not need to generate a text animation video;

[0107] "BG_FILE" indicates the background file of the storyboard, which can be a picture or a video;

[0108] "DH_POSITION" indicates the position of the digital human in the storyboard video, which can be set to "center / right / left / lower left / lower right" and other forms;

[0109] "DH_SHOW" indicates the style of the digital human presentation. "with_bg" indicates that the image needs to be cut out using the cutout algorithm and the background is integrated according to the method in "DH_POSITION" to generate a storyboard video. If it is empty, it means that no cutout is required and the digital human video generated by the original digital human driving algorithm is directly used as the storyboard video.

[0110] The "DH_FILE" value refers to the name of the digital human image used by the relevant storyboard in the digital human image library, which is used to generate the digital human animation video in the storyboard;

[0111] "TRANSITION" indicates the transition animation between the current video and the next video, and the value is the name of the transition animation.

[0112] The above video configuration is specifically shown in Table 2, which is mainly used to set the relevant parameters of the final output video, such as height, width, whether to add subtitles, background sound files, etc.

[0113] Table 2 Video Configuration Table

[0114] Settings value Remark Video width 1280 Video High 720 subtitle none Background Sound bgm.mp3

[0115] The above-mentioned voice configuration is specifically shown in Table 3, which is mainly used for setting relevant parameters in the speech synthesis process, such as the speaker's name, speaking speed, volume and audio sampling rate.

[0116] Table 3 Voice Configuration Table

[0117] Settings value Remark voice_id tiantian Pronunciation Role speed 0.9 Speech rate volume 1 volume audio_rate 16000 Sampling rate

[0118] If the above method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0119] Example 2

[0120] Based on the specific implementation steps of a digital human video generation method based on AIGC technology disclosed in Example 1, the present invention provides a digital human video generation device based on AIGC technology, the device comprising:

[0121] The material receiving module receives character pictures, configuration tables, material files and background materials. The configuration table includes storyboard configuration, video configuration and voice configuration.

[0122] The material processing module uses AIGC technology to process character images, build a digital human image library, pre-process the material files, and convert them into image files. Each image in the image file corresponds to a storyboard in the storyboard configuration;

[0123] The video generation module synthesizes the broadcast voice and subtitle files according to the storyboard configuration and voice configuration, extracts the digital human image from the digital human image library according to the storyboard configuration, extracts the voice features from the broadcast voice and drives the digital human image by using the digital human driving technology, extracts the voice features from the broadcast voice, drives the digital human image, and generates a digital human broadcast video that matches the broadcast voice;

[0124] The video editing module integrates background materials and picture files of each storyboard, broadcast voice, subtitle files and digital human broadcast video, and performs micro-editing according to the configuration table;

[0125] The video output module outputs the final digital human video and uploads it to the S3 storage bucket.

[0126] This device can focus on 2D digital human application scenarios based on AIGC image technology, and is committed to using templated functional design ideas to complete multimedia stream combination tasks and generate digital human presentation videos. The specific details of the above modules can be understood by referring to the relevant descriptions and effects in Example 1.

[0127] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A digital human video generation method based on AIGC technology, It is characterized in that The method comprises: Obtain character pictures, configuration tables, material files and background materials. The configuration table includes storyboard configuration, video configuration and voice configuration; Use AIGC image technology to process human images and build a digital human image library; Preprocessing the material file and converting it into an image file, wherein each image in the image file corresponds to a storyboard in the storyboard configuration; According to the storyboard configuration and voice configuration, synthesize the broadcast voice and subtitle files according to the storyboard; Extract digital human images from the digital human image library according to the storyboard configuration, extract voice features from the broadcast voice using digital human driving technology and drive the digital human images to generate a digital human broadcast video that matches the broadcast voice; Combine the background material and the picture files of each storyboard, the broadcast voice, the subtitle files and the digital human broadcast video, perform micro-editing according to the configuration table, and output the final digital human video.

2. According to the method for generating digital human video based on AIGC technology in claim 1, It is characterized in that The mirror configuration includes: INDEX, storyboard number; TEXT: the text that needs to be broadcast or displayed in the storyboard; TYPE, whether the storyboard needs to generate text animation video; BG_FILE, storyboard background file, the formats include pictures and videos; DH_POSITION, the position of the digital human in the storyboard video, including center, right, left, lower left, and lower right; DH_SHOW, the presentation style of the storyboard digital human; DH_FILE, digital human image extracted from the digital human image library; TRANSITION, the transition animation between the current storyboard and the next storyboard.

3. The method for generating digital human video based on AIGC technology according to claim 1, It is characterized in that The video configuration is used to set the parameters used in the final digital human video output process, including the height and width of the video, whether to add subtitles and background sound files, and the names of the added subtitle files and background sound files.

4. The method for generating digital human video based on AIGC technology according to claim 1, It is characterized in that The speech configuration is used to set the parameters used in the speech synthesis process, including the pronunciation role name, speaking speed, volume and audio sampling rate.

5. The method for generating digital human video based on AIGC technology according to claim 1, It is characterized in that The step of processing the character image using the AIGC technology comprises: Screening the person pictures, retaining the pictures with the same size, clear boundaries between the background and the person, and different actions of the person that meet the action display requirements in different scenes, wherein the action display requirements include the finger pointing to the left and the finger pointing to the right; Name the filtered pictures with the action name; Use AIGC image technology to segment the screened and named person images into different blocks, retain the clothing area, and generate clothing masks; Fill in the positive and negative prompts according to the presentation style of the digital human in the storyboard configuration, and use AIGC technology to combine the image and the clothing mask to redraw the image within the mask to generate a picture of the character after changing clothes; Using AIGC image technology, the poses of the characters after changing clothes are guided to generate characters with different actions; Filter the generated images and name them after the action; Output the digital human image after the change of clothes and the corresponding original image.

6. The method for generating digital human video based on AIGC technology according to claim 1, It is characterized in that The pre-processing comprises: If the source file is in PPT format, convert the source file into pictures page by page, and rename the pictures in order according to the storyboard numbers in the storyboard configuration; If the material file is a picture, it is named in the order of display number, and the picture order is renamed according to the storyboard number in the storyboard configuration; If the source file is in word format, convert the file to pdf format and then convert it into pictures page by page, and rename the pictures in order according to the storyboard numbers in the storyboard configuration; If the source file is in PDF format, convert the file into images page by page, and rename the images in order according to the storyboard numbers in the storyboard configuration; Organize the renamed pictures and output the picture files.

7. The method for generating digital human video based on AIGC technology according to claim 1, It is characterized in that The steps of synthesizing the broadcast voice and subtitle files include: Set the speaker's timbre, speaking speed and sampling rate according to the voice configuration; According to the storyboard configuration, speech synthesis technology is used to synthesize the broadcast voice and subtitle files containing timestamps.

8. The method for generating digital human video based on AIGC technology according to claim 2, It is characterized in that The micro-clipping includes: Obtain the storyboard type parameters in the storyboard configuration, combine the picture file, broadcast voice, subtitle file and digital human broadcast video, and generate each storyboard video; According to the storyboard configuration, add the specified transition animation to the storyboard room; Use video processing tools to connect storyboard videos in sequence; According to the video configuration, add background sound to the video and synthesize the final digital human video.

9. The method for generating digital human video based on AIGC technology according to claim 8, It is characterized in that The storyboard type parameters are TEXT and DH_SHOW in the storyboard configuration, where: When TEXT is text, it means generating a 3s text animation video according to the text that needs to be broadcast or displayed in the storyboard configuration corresponding to the storyboard; When TEXT is empty, it means that the storyboard does not need to generate a text animation video; When DH_SHOW is with_bg, it means extracting the digital human broadcast video with a transparent background of the storyboard, and then combining the digital human broadcast video with a transparent background of the storyboard, the material picture and the background material to generate a storyboard video with the same duration as the corresponding broadcast voice; When DH_SHOW is only, it means extracting the digital human broadcast video with a transparent background of the storyboard, and then combining the digital human broadcast video with a transparent background of the storyboard with the background material to generate a storyboard video with the same duration as the corresponding broadcast voice; When DH_SHOW is empty, it means generating a storyboard video that only displays the background material and the storyboard image file.

10. A digital human video generation device based on AIGC technology, It is characterized in that The device comprises: The material receiving module receives character pictures, configuration tables, material files and background materials. The configuration table includes storyboard configuration, video configuration and voice configuration. The material processing module uses AIGC image technology to process character images, build a digital human image library, pre-process the material files, and convert them into image files. Each image in the image file corresponds to a storyboard in the storyboard configuration; The video generation module synthesizes the broadcast voice and subtitle files according to the storyboard configuration and voice configuration, extracts the digital human image from the digital human image library according to the storyboard configuration, extracts the voice features from the broadcast voice and drives the digital human image by using the digital human driving technology, extracts the voice features from the broadcast voice, drives the digital human image, and generates a digital human broadcast video that matches the broadcast voice; The video editing module integrates background materials and picture files of each storyboard, broadcast voice, subtitle files and digital human broadcast video, and performs micro-editing according to the configuration table; The video output module outputs the final digital human video and uploads it for storage.

Citation Information

Cited By

  • Agent cooperation-based AIGC long video stable generation method and system

    CN119893206A

  • An AIGC long video stabilization generation method and system based on agent cooperation

    CN119893206B

  • SSML text automatic generation method and device fusing sub-mirror hierarchy information

    CN121072489A

  • Method and apparatus for automatically generating ssml text fusing split camera level information

    CN121072489B