Digital human broadcast video generation method and system

Abstract

The invention provides a digital people broadcast video generation method and system, and the method comprises the steps: obtaining an original material set, and carrying out the format conversion of materials in the original material set, so as to obtain a converted material set; performing audio processing on the converted audio to obtain a processed audio; performing digital population reasoning on the converted picture and the converted video based on the processed audio to obtain a digital human sequence frame; and determining a background video based on the converted picture, and carrying out video layer superposition on the background video and the digital human sequence frame to obtain a digital human broadcast video. The method distinguishes speaking and non-speaking through audio analysis, is compatible with various digital mouth shape reasoning technologies, and detects the face shape through face detection, thereby selecting a more suitable mouth shape reasoning technology; and meanwhile, the whole broadcast video generation process is developed, so that the conditions of convulsion, beard erasing and face deformation of the digital population can be avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of image restoration, and in particular relates to a method and system for generating a digital human broadcast video. Background Art

[0002] At present, the way digital human broadcast videos are generated is basically a combination of manual and part AI generation. Especially for some complex broadcast videos, the higher the degree of manual participation, the more time and labor costs will be generated.

[0003] However, the digital human broadcasting video in the prior art has the following problems: due to the large number of material formats, it is difficult to use them in synthesis; after the video is generated through audio, the digital human mouth is prone to twitching when not speaking; the digital human broadcasting video is prone to having its beard erased and the human face deformed. Summary of the invention

[0004] In order to solve the above technical problems, the present invention provides a method and system for generating a digital human broadcast video, which is used to solve the technical problems raised in the prior art, so as to reduce the user's time cost and ensure that the user can find the target terminal in the shortest time.

[0005] In a first aspect, an embodiment of the present invention provides the following technical solution, a method for generating a digital human broadcast video, comprising:

[0006] Acquire an original material set, and perform format conversion on materials in the original material set to obtain a converted material set, wherein the converted material set includes converted pictures, converted audio, and converted video;

[0007] Performing audio processing on the converted audio to obtain processed audio;

[0008] Performing digital human type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame;

[0009] A background video is determined based on the conversion image, and the background video and the digital human sequence frames are superimposed in video layers to obtain a digital human broadcast video.

[0010] Compared with the prior art, the beneficial effects of the present application are as follows: the present invention first obtains an original material set, performs format conversion on the materials in the original material set to obtain a converted material set, the converted material set includes converted pictures, converted audio and converted video; then performs audio processing on the converted audio to obtain processed audio; then performs digital population type inference on the converted picture and the converted video based on the processed audio to obtain a digital human sequence frame; finally, a background video is determined based on the converted picture, and a video layer is superimposed on the background video and the digital human sequence frame to obtain a digital human broadcast video. The present invention uses audio analysis to distinguish between speaking and not speaking, and is compatible with wav2lip and musetalk digital population type inference technologies. It detects face shape through face detection, thereby selecting a more suitable lip shape inference technology, and develops the entire broadcast video generation process at the same time, and changes the background, adds picture-in-picture, foreground image, foreground text, subtitles, watermark and other layers with one click, which can avoid twitching, beard wiping and face deformation of the digital population type.

[0011] Preferably, the step of obtaining an original material set and converting the format of the materials in the original material set to obtain a converted material set includes:

[0012] Acquire an original material set, wherein the original material set includes original picture materials, original audio materials, and original video materials;

[0013] The original image video is encoded using opencv and yuv420p in jpg or png format to obtain a converted image;

[0014] Encoding the original audio material using the wav format aac to obtain converted audio;

[0015] The original video material is encoded using ffmpeg and mp4 format h264 to obtain a converted video.

[0016] Preferably, the step of performing audio processing on the converted audio to obtain the processed audio is specifically:

[0017] The converted audio is input into ffmpeg, and the time interval of the converted audio is extracted through ffmpeg, the amplitude of the audio corresponding to each time interval is identified, the time area corresponding to the amplitude less than the preset value is eliminated, and the audio corresponding to the remaining time intervals is merged in sequence to obtain the processed audio.

[0018] Preferably, the step of performing digital human population type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame comprises:

[0019] Using a preset recognition algorithm to perform face recognition on the conversion image and the conversion video to obtain a set of facial feature points;

[0020] Calculate the face shape ratio R based on the face feature point set:

[0021] V 2,14 =(V2-V 14 ) / 2;

[0022] R=distance(V7,V 2,14 ) / dostance(V2,V 2,14 );

[0023] Where V 2,14 Indicates the second facial feature point V2 and the fourteenth facial feature point V in the facial feature point set 14 The midpoint between V7 and V 2,14 ) indicates that V7 and V 2,14 The distance between them, distance(V2,V 2,14 ) indicates that V2 and V 2,14 The distance between

[0024] If the face shape ratio R is not greater than the preset ratio, the digital population type inference is performed on the conversion image and the conversion video based on the processed audio and using the first inference algorithm; if the face shape ratio R is greater than the preset ratio, the digital population type inference is performed on the conversion image and the conversion video based on the processed audio and using the second inference algorithm to obtain a digital human sequence frame.

[0025] Preferably, the preset recognition algorithm is specifically the shape_predictor_68_face_landmarks.dat algorithm.

[0026] Preferably, the first inference algorithm is a wav2lip algorithm, and the second inference algorithm is a musetalk algorithm.

[0027] Preferably, the step of determining the background video based on the conversion image, and superimposing the background video and the digital human sequence frames in video layers to obtain the digital human broadcast video comprises:

[0028] Extracting a background image of the conversion image, and generating a background video having the same duration as the conversion audio based on the background image;

[0029] Input the background video into ffmpeg and input the cropping, scaling, position and rotation parameters;

[0030] The picture-in-picture, the digital human sequence frames, the foreground image, the text, the subtitles and the watermark are input into ffmpeg for layer-by-layer superposition to obtain the digital human broadcasting video.

[0031] In a second aspect, an embodiment of the present invention provides the following technical solution: a digital human broadcast video generation system, the system comprising:

[0032] A conversion module, used to obtain an original material set, and perform format conversion on the materials in the original material set to obtain a converted material set, wherein the converted material set includes converted pictures, converted audio, and converted video;

[0033] A processing module, used for performing audio processing on the converted audio to obtain processed audio;

[0034] An inference module, configured to perform digital human type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame;

[0035] The superposition module is used to determine the background video based on the conversion image, and to perform video layer superposition on the background video and the digital human sequence frames to obtain the digital human broadcast video.

[0036] In a third aspect, an embodiment of the present invention provides the following technical solution: a computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned method for generating a digital human broadcast video when executing the computer program.

[0037] In a fourth aspect, an embodiment of the present invention provides the following technical solution: a computer-readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the above-mentioned method for generating a digital human broadcast video. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 A flowchart of a method for generating a digital human broadcast video provided by an embodiment of the present invention;

[0040] Figure 2 A structural block diagram of a digital human broadcast video generation system provided by an embodiment of the present invention;

[0041] Figure 3A schematic diagram of the hardware structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the embodiments of the present invention, and should not be construed as limiting the present invention.

[0043] Embodiment 1

[0044] In the first embodiment of the present invention, Figure 1 As shown, a method for generating a digital human broadcast video includes:

[0045] S1, obtaining an original material set, and performing format conversion on materials in the original material set to obtain a converted material set, wherein the converted material set includes converted pictures, converted audio, and converted video;

[0046] Specifically, since the original material set includes a variety of different materials, and the formats of different materials vary, in order to ensure that the original material set can be smoothly put into use in video generation, it is necessary to convert the format of the material to facilitate the subsequent video generation process.

[0047] Wherein, the step S1 comprises:

[0048] S11. Acquire an original material set, where the original material set includes original picture materials, original audio materials, and original video materials.

[0049] S12, using opencv and yuv420p encoding in jpg or png format to obtain a converted image.

[0050] S13, encoding the original audio material using the wav format aac to obtain converted audio.

[0051] S14, using ffmpeg and using the mp4 format h264 encoding on the original video material to obtain a converted video;

[0052] Specifically, ffmpeg is an open source streaming media processing library with a powerful API. It can produce videos in corresponding formats by simply inputting corresponding materials and parameters, and opencv is a commonly used processing tool in the prior art, so it will not be described here.

[0053] S2, performing audio processing on the converted audio to obtain processed audio;

[0054] Wherein, the step S2 is specifically as follows:

[0055] Input the converted audio into ffmpeg, extract the time interval of the converted audio through ffmpeg, identify the amplitude of the audio corresponding to each time interval, remove the time area corresponding to the amplitude less than the preset value, and merge the audio corresponding to the remaining time intervals in sequence to obtain the processed audio;

[0056] Specifically, in this step, the preset value is 0.05. When the amplitude of the audio in a certain time interval is less than 0.05, the audio can be judged as silent audio, that is, audio without speaking. The silent audio is then eliminated and only the speaking audio is retained. The audio is then synthesized in sequence to obtain the processed audio. This can avoid the situation where the digital population twitches even when it is not speaking.

[0057] S3, performing digital human type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame;

[0058] Wherein, the step S3 comprises:

[0059] S31, using a preset recognition algorithm to perform face recognition on the conversion image and the conversion video to obtain a set of facial feature points;

[0060] Specifically, the preset recognition algorithm is specifically the shape_predictor_68_face_landmarks.dat algorithm. After the feature points are recognized by the preset recognition algorithm, the feature points are marked and numbered in the image or video.

[0061] S32, calculating the face shape ratio R based on the face feature point set:

[0062] V 2,14 =(V2-V 14 ) / 2;

[0063] R=distance(V7,V 2,14 ) / distance(V2,V 2,14 );

[0064] Where V 2,14 Indicates the second facial feature point V2 and the fourteenth facial feature point V in the facial feature point set 14 The midpoint between V7 and V 2,14 ) indicates that V7 and V 2,14 The distance between them, distance(V2,V 2,14 ) indicates that V2 and V 2,14 The distance between.

[0065] S33, if the face shape ratio R is not greater than the preset ratio, then based on the processed audio and using the first inference algorithm, digital population type inference is performed on the conversion image and the conversion video; if the face shape ratio R is greater than the preset ratio, then based on the processed audio and using the second inference algorithm, digital population type inference is performed on the conversion image and the conversion video to obtain a digital human sequence frame;

[0066] Specifically, the preset ratio here is 0.8. When the face shape ratio is not greater than 0.8, it means that the face is round, and the present application adopts the first reasoning algorithm for reasoning, and the first reasoning algorithm is the wav2lip algorithm. The wav2lip algorithm is suitable for round faces and can maintain the details of the face, but is not suitable for melon-seed faces and digital people who need skin smoothing. When the face shape ratio is greater than 0.8, it means that the face is melon-seed faces, and the present application adopts the second reasoning algorithm for reasoning, and the second reasoning algorithm is the musetalk algorithm. The musetalk algorithm is suitable for digital people who are suitable for melon-seed faces and need skin smoothing. Therefore, the present invention identifies the specific face shape of the face and then adapts to different reasoning algorithms, thereby avoiding the shortcomings of these two reasoning technologies, and promoting their strengths, and then being compatible with a variety of lip reasoning technologies to better adapt to various face shapes, avoiding situations such as beards being wiped off and faces being deformed.

[0067] S4. Determine a background video based on the conversion image, and perform video layer superposition on the background video and the digital human sequence frames to obtain a digital human broadcast video.

[0068] Wherein, the step S4 comprises:

[0069] S41, extracting the background image of the conversion image, and generating a background video having the same duration as the conversion audio based on the background image.

[0070] S42, input the background video into ffmpeg, and input cropping, scaling, position, and rotation parameters.

[0071] S43, inputting the picture-in-picture, the digital human sequence frames, the foreground image, the text, the subtitles, and the watermark into ffmpeg for layer-by-layer superposition to obtain a digital human broadcast video;

[0072] Specifically, the above-mentioned picture-in-picture, the digital human sequence frame, foreground image, text, subtitle, and watermark can be superimposed on the background video layer by layer to obtain the digital human broadcast video. Therefore, this solution can expand the full range of high-quality digital human mouth reasoning by being compatible with more lip-sync reasoning technologies. This solution can change the superposition order of each video layer to change the generated level. This solution can be used as a service for any platform that needs to produce digital human broadcast videos. This solution is a one-click generation, which reduces manpower investment and saves time, and can effectively reduce the investment cost of the enterprise. This solution takes some abnormal factors into consideration at each step, and makes the best choice by detecting parameters to avoid producing unqualified videos.

[0073] The method for generating a digital human broadcast video provided in the first embodiment of the present invention first obtains an original material set, performs format conversion on the materials in the original material set to obtain a converted material set, wherein the converted material set includes a converted picture, a converted audio, and a converted video; then performs audio processing on the converted audio to obtain a processed audio; then performs digital population type inference on the converted picture and the converted video based on the processed audio to obtain a digital human sequence frame; finally, a background video is determined based on the converted picture, and video layers are superimposed on the background video and the digital human sequence frame to obtain a digital human broadcast video. The present invention distinguishes between speaking and not speaking through audio analysis, and is compatible with wav2lip and musetalk digital population type inference technologies, detects face shape through face detection, and thus selects a more appropriate lip shape inference technology, and simultaneously develops the entire broadcast video generation process, and changes the background, adds picture-in-picture, foreground image, foreground text, subtitles, watermark and other layers with one click, which can avoid twitching, beard erasing and face deformation of the digital population type.

[0074] Embodiment 2

[0075] like Figure 2 As shown, in the second embodiment of the present invention, a digital human broadcast video generation system is provided, and the system includes:

[0076] The conversion module 1 is used to obtain an original material set, and perform format conversion on the materials in the original material set to obtain a converted material set, wherein the converted material set includes converted pictures, converted audio, and converted video;

[0077] Processing module 2, used for performing audio processing on the converted audio to obtain processed audio;

[0078] An inference module 3, configured to perform digital human type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame;

[0079] The superposition module 4 is used to determine the background video based on the conversion picture, and to perform video layer superposition on the background video and the digital human sequence frame to obtain the digital human broadcast video;

[0080] The conversion module 1 is specifically used for:

[0081] An acquisition submodule, used to acquire an original material set, wherein the original material set includes original picture materials, original audio materials and original video materials;

[0082] The first conversion submodule is used to encode the original image video using opencv and yuv420p in jpg or png format to obtain a converted image;

[0083] The second conversion submodule is used to encode the original audio material using the wav format aac to obtain the converted audio;

[0084] The third conversion submodule is used to use ffmpeg and mp4 format h264 to encode the original video material to obtain a converted video.

[0085] The processing module 2 is specifically used for:

[0086] The converted audio is input into ffmpeg, and the time interval of the converted audio is extracted through ffmpeg, the amplitude of the audio corresponding to each time interval is identified, the time area corresponding to the amplitude less than the preset value is eliminated, and the audio corresponding to the remaining time intervals is merged in sequence to obtain the processed audio.

[0087] The reasoning module 3 comprises:

[0088] A recognition submodule, used for performing face recognition on the conversion image and the conversion video using a preset recognition algorithm to obtain a set of facial feature points;

[0089] A calculation submodule is used to calculate the face shape ratio R based on the face feature point set:

[0090] V 2,14 =(V2-V 14 ) / 2;

[0091] R=distance(V7,V 2,14 ) / distance(V2,V 2,14 );

[0092] Where V 2,14 Indicates the second facial feature point V2 and the fourteenth facial feature point V in the facial feature point set 14 The midpoint between V7 and V2,14 ) indicates that V7 and V 2,14 The distance between them, distance(V2,V 2,14 ) indicates that V2 and V 2,14 The distance between

[0093] The inference submodule is used to perform digital population type inference on the conversion image and the conversion video based on the processed audio and using a first inference algorithm if the face shape ratio R is not greater than a preset ratio; if the face shape ratio R is greater than the preset ratio, perform digital population type inference on the conversion image and the conversion video based on the processed audio and using a second inference algorithm to obtain a digital human sequence frame.

[0094] The superposition module 4 comprises:

[0095] An extraction submodule, used for extracting a background image of the conversion image, and generating a background video with the same duration as the conversion audio based on the background image;

[0096] The parameter submodule is used to input the background video into ffmpeg and input cropping, scaling, position, and rotation parameters;

[0097] The superposition submodule is used to input the picture-in-picture, the digital human sequence frame, the foreground image, the text, the subtitle, and the watermark into ffmpeg for layer-by-layer superposition to obtain the digital human broadcast video.

[0098] In some other embodiments of the present invention, the embodiments of the present invention provide the following technical solutions: a computer, comprising a memory 102, a processor 101, and a computer program stored in the memory 102 and executable on the processor 101; when the processor 101 executes the computer program, the method for generating a digital human broadcast video as described above is implemented.

[0099] Specifically, the processor 101 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiment of the present invention.

[0100] Among them, the memory 102 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 102 may include a hard disk drive (HDD), a floppy disk drive, a solid state drive (SSD), a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive, or a combination of two or more of these. Where appropriate, the memory 102 may include a removable or non-removable (or fixed) medium. Where appropriate, the memory 102 may be inside or outside a data processing device. In a specific embodiment, the memory 102 is a non-volatile memory. In a specific embodiment, the memory 102 includes a read-only memory (ROM) and a random access memory (RAM). Where appropriate, the ROM may be a mask-programmed ROM, a programmable ROM (Programmable Read-Only Memory, PROM for short), an erasable PROM (ErasableProgrammable Read-Only Memory, EPROM for short), an electrically erasable PROM (Electrically ErasableProgrammable Read-Only Memory, EEPROM for short), an electrically alterable ROM (ElectricallyAlterable Read-Only Memory, EAROM for short) or a flash memory (FLASH) or a combination of two or more of these. Under appropriate circumstances, the RAM can be a static random access memory (SRAM) or a dynamic random access memory (DRAM), wherein the DRAM can be a fast page mode dynamic random access memory (FPMDRAM), an extended data output dynamic random access memory (EDODRAM), a synchronous dynamic random access memory (SDRAM), etc.

[0101] The memory 102 may be used to store or cache various data files that need to be processed and / or used for communication, as well as possible computer program instructions executed by the processor 101 .

[0102] The processor 101 implements the above-mentioned method for generating a digital human broadcast video by reading and executing the computer program instructions stored in the memory 102 .

[0103] In some embodiments, the computer may further include a communication interface 103 and a bus 100. Figure 3 As shown, the processor 101, the memory 102, and the communication interface 103 are connected via a bus 100 and communicate with each other.

[0104] The communication interface 103 is used to implement communication between the modules, devices, units and / or equipment in the embodiment of the present invention. The communication interface 103 can also implement data communication with other components such as: external devices, image / data acquisition equipment, databases, external storage, and image / data processing workstations.

[0105] The bus 100 includes hardware, software or both, and couples the components of the computer device to each other. The bus 100 includes but is not limited to at least one of the following: a data bus, an address bus, a control bus, an expansion bus, and a local bus. By way of example and not limitation, bus 100 may include an Accelerated Graphics Port (AGP) or other graphics bus, an Extended Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a Hyper Transport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association Local Bus (VLB) bus, or other suitable buses or a combination of two or more of the above. Where appropriate, bus 100 may include one or more buses. Although embodiments of the present invention describe and illustrate a particular bus, the present invention contemplates any suitable bus or interconnect.

[0106] The computer can execute the digital human broadcast video generation method of the present invention based on the acquired digital human broadcast video generation system, thereby realizing the digital human broadcast video generation.

[0107] In some further embodiments of the present invention, in combination with the above-mentioned method for generating a digital human broadcast video, the embodiments of the present invention provide the following technical solutions: a storage medium having a computer program stored thereon, and the computer program implements the above-mentioned method for generating a digital human broadcast video when executed by a processor.

[0108] Those skilled in the art will appreciate that the logic and / or steps represented in the flowchart or otherwise described herein, for example, may be considered as an ordered list of executable instructions for implementing logical functions, and may be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For purposes of this specification, a "computer-readable medium" may be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0109] More specific examples of readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0110] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or a combination thereof: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0111] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0112] The above-mentioned embodiments only express several implementation methods of the present invention, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent of the present invention shall be subject to the attached claims.

Claims

1. A method for generating a digital human broadcast video, characterized in that: include: Acquire an original material set, and perform format conversion on materials in the original material set to obtain a converted material set, wherein the converted material set includes converted pictures, converted audio, and converted video; Performing audio processing on the converted audio to obtain processed audio; Performing digital human type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame; A background video is determined based on the conversion image, and the background video and the digital human sequence frames are superimposed in video layers to obtain a digital human broadcast video.

2. The method for generating a digital human broadcast video according to claim 1, characterized in that: The step of obtaining an original material set and converting the format of the materials in the original material set to obtain a converted material set includes: Acquire an original material set, wherein the original material set includes original picture materials, original audio materials, and original video materials; The original image video is encoded using opencv and yuv420p in jpg or png format to obtain a converted image; Encoding the original audio material using the wav format aac to obtain converted audio; The original video material is encoded using ffmpeg and mp4 format h264 to obtain a converted video.

3. The method for generating a digital human broadcast video according to claim 1, characterized in that: The step of performing audio processing on the converted audio to obtain processed audio is specifically: The converted audio is input into ffmpeg, and the time interval of the converted audio is extracted through ffmpeg, the amplitude of the audio corresponding to each time interval is identified, the time area corresponding to the amplitude less than the preset value is eliminated, and the audio corresponding to the remaining time intervals is merged in sequence to obtain the processed audio.

4. The method for generating a digital human broadcast video according to claim 1, characterized in that: The step of performing digital human population type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame comprises: Using a preset recognition algorithm to perform face recognition on the conversion image and the conversion video to obtain a set of facial feature points; Calculate the face shape ratio R based on the face feature point set: V 2,14 =(V2-V 14 ) / 2; R=distance(V7,V 2,14 ) / distance(V2,V 2,14 ); Where V 2,14 Indicates the second facial feature point V2 and the fourteenth facial feature point V in the facial feature point set 14 The midpoint between V7 and V 2,14 ) indicates that V7 and V 2,14 The distance between them, distance(V2,V 2,14 ) indicates that V2 and V 2,14 The distance between If the face shape ratio R is not greater than the preset ratio, the digital population type inference is performed on the conversion image and the conversion video based on the processed audio and using the first inference algorithm; if the face shape ratio R is greater than the preset ratio, the digital population type inference is performed on the conversion image and the conversion video based on the processed audio and using the second inference algorithm to obtain a digital human sequence frame.

5. The method for generating a digital human broadcast video according to claim 4, characterized in that: The preset recognition algorithm is specifically the shape_predictor_68_face_landmarks.dat algorithm.

6. The method for generating a digital human broadcast video according to claim 4, characterized in that: The first inference algorithm is the wav2 lip algorithm, and the second inference algorithm is the musetalk algorithm.

7. The method for generating a digital human broadcast video according to claim 1, characterized in that: The step of determining the background video based on the conversion image, and superimposing the background video and the digital human sequence frames by video layers to obtain the digital human broadcast video comprises: Extracting a background image of the conversion image, and generating a background video having the same duration as the conversion audio based on the background image; Input the background video into ffmpeg and input the cropping, scaling, position and rotation parameters; The picture-in-picture, the digital human sequence frames, the foreground image, the text, the subtitles and the watermark are input into ffmpeg for layer-by-layer superposition to obtain the digital human broadcasting video.

8. A digital human broadcast video generation system, characterized in that: include: A conversion module, used to obtain an original material set, and perform format conversion on the materials in the original material set to obtain a converted material set, wherein the converted material set includes converted pictures, converted audio, and converted video; A processing module, used for performing audio processing on the converted audio to obtain processed audio; An inference module, configured to perform digital human type inference on the converted image and the converted video based on the processed audio to obtain a digital human sequence frame; The superposition module is used to determine the background video based on the conversion image, and to perform video layer superposition on the background video and the digital human sequence frames to obtain the digital human broadcast video.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for generating a digital human broadcast video according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for generating a digital human broadcast video according to any one of claims 1 to 7 is implemented.

Images