A method, device, equipment and storage medium for generating virtual human video

Abstract

The embodiment of the present invention discloses a method, device, equipment and storage medium for generating a virtual human video. The method comprises: obtaining input content; wherein the input content includes a dialogue text and a video script, which is used to generate the dubbing and picture of the video; determining at least one moving time point based on the dialogue text and the video script; adding at least one lower body moving animation at each moving time point to obtain an animation set corresponding to each moving time point; wherein the moving time point is the starting time point of the lower body moving animation playback; determining a target lower body moving animation sequence based on the animation set corresponding to each moving time point; generating a virtual human full body moving video according to the target lower body moving animation sequence and the input content. The method for generating a virtual human video provided by the embodiment of the present invention can prevent the problems of uncontrollable walking and sliding of the virtual human in the generated video, so as to improve the generation quality and display effect of the virtual human video.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of video generation, and in particular to a method, device, equipment and storage medium for generating a virtual human video. Background Art

[0002] With the introduction of the concept of "metaverse", virtual people are gaining more and more attention in the fields of e-commerce, games and animation, and their influence is gradually expanding. For example, in live broadcast videos, virtual people can replace real people to introduce and explain products, which can not only save labor costs, but also bind virtual people to products to avoid losses in product sales caused by the resignation of the anchor. At product launch conferences, virtual people can replace real people to introduce products, which can not only attract the attention of the audience, but also achieve long-term uninterrupted high-quality repeated introductions.

[0003] In order to make virtual people replace real people and gain the trust of the audience, the virtual people need to be as close to real people in speech and behavior as possible. In addition to being able to make the same voice as a real person, they also need to make corresponding gestures, expressions, movements, and move or walk in the video.

[0004] In the related technology, a virtual human video can be automatically generated based on audio input. The walking or movement of the virtual human in the video is completely controlled by the video generation software, which is different from the behavior habits of real people. In addition, the virtual human is prone to slipping when moving or walking, that is, the feet of the virtual human slide unnaturally on the ground, which seriously affects the video quality and display effect. Summary of the invention

[0005] The embodiments of the present invention provide a method, device, equipment and storage medium for generating a virtual human video, which can prevent problems such as uncontrollable walking and sliding of the virtual human in the generated video, so as to improve the generation quality and display effect of the virtual human video.

[0006] In a first aspect, an embodiment of the present invention provides a method for generating a virtual human video, comprising:

[0007] Obtaining input content; wherein the input content includes dialogue text and video script, which is used to generate the dubbing and picture of the video;

[0008] Determining at least one moving time point based on the dialogue text and the video script;

[0009] Add at least one lower body movement animation at each movement time point to obtain an animation set corresponding to each movement time point; wherein the movement time point is the starting time point of playing the lower body movement animation;

[0010] A target lower body movement animation sequence is determined based on an animation set corresponding to each of the movement time points; and a virtual human full body movement video is generated according to the target lower body movement animation sequence and the input content.

[0011] In a second aspect, an embodiment of the present invention further provides a device for generating a virtual human video, comprising:

[0012] An input content acquisition module, used to acquire input content; wherein the input content includes dialogue text and video script, which is used to generate the dubbing and picture of the video;

[0013] A moving time point determination module, used to determine at least one moving time point based on the dialogue text and the video script;

[0014] A lower body movement animation adding module, used to add at least one lower body movement animation at each movement time point, and obtain an animation set corresponding to each movement time point; wherein the movement time point is the starting time point of the lower body movement animation;

[0015] A target lower body movement animation sequence determination module, used to determine the target lower body movement animation sequence based on the animation set corresponding to each movement time point;

[0016] The virtual human full body movement video generation module is used to generate a virtual human full body movement video according to the target lower body movement animation sequence and the input content.

[0017] In a third aspect, an embodiment of the present invention further provides an electronic device, the electronic device comprising:

[0018] at least one processor; and a memory in communication with the at least one processor; wherein,

[0019] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for generating a virtual human video described in an embodiment of the present invention.

[0020] In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for generating a virtual human video described in the embodiment of the present invention when executed.

[0021] In a fifth aspect, an embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for generating a virtual human video as described in the embodiment of the present invention.

[0022] The embodiment of the present invention discloses a method, device, equipment and storage medium for generating a virtual human video. Input content is obtained; wherein the input content includes a line text and a video script, which is used to generate the dubbing and picture of the video; at least one moving time point is determined based on the line text and the video script; at least one lower body moving animation is added to each moving time point to obtain an animation set corresponding to each moving time point; wherein the moving time point is the starting time point of the lower body moving animation playback; based on the animation set corresponding to each moving time point, a target lower body moving animation sequence is determined; and a virtual human full body moving video is generated according to the target lower body moving animation sequence and the input content. The method for generating a virtual human video provided by an embodiment of the present invention first determines at least one movement time point based on a dialogue text and a video script, then determines a target lower body movement animation sequence by using the lower body animation added at each movement time point, and then generates a virtual human full body movement video based on the target lower body movement animation sequence and input content, thereby achieving precise control over the time point of the virtual human movement, the lower body movement animation, the full body movement, etc., and preventing problems such as uncontrollable walking and sliding of the virtual human from occurring in the generated video, thereby improving the generation quality and display effect of the virtual human video. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a method for generating a virtual human video in an embodiment of the present invention;

[0024] Figure 2 This is an example diagram of dividing the time axis into a movable period, an immovable period, and a pending movement period in this embodiment;

[0025] Figure 3 This is an example diagram of adding lower body movement animation at the moving time point in this embodiment;

[0026] Figure 4 is a structural schematic diagram of a device for generating a virtual human video in an embodiment of the present invention;

[0027] Figure 5 It is a structural schematic diagram of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

[0029] Figure 1A flowchart of a method for generating a virtual human video is provided in Embodiment 1 of the present invention. This embodiment is applicable to the case of generating a video of a virtual human moving or walking. The method can be executed by a video generating device for a virtual human, which can be implemented in the form of software and / or hardware, and optionally, by an electronic device, which can be a mobile terminal, a PC or a server, etc.

[0030] Among them, the virtual person can also be called a virtual digital person, for example: a virtual news anchor, a virtual teacher, a virtual anchor, etc., and this embodiment does not limit the application scenario of the virtual person. The virtual person in this embodiment can be a two-dimensional character image, a three-dimensional character model, and a character model of more dimensions, and this embodiment does not limit this. Through the scheme of this embodiment, the user inputs at least one paragraph of text, and after generating audio according to the text, a video of a virtual person explaining the text can be quickly generated, for example: it can be a news broadcast video, a product introduction video, and a knowledge popularization video. In addition, in order to enhance the display effect of the video, this embodiment also allows the user to input some video materials, such as images or videos, while inputting the aforementioned text. The images or videos can be used as the display content when the virtual person explains in the generated video. Further, in order to facilitate user operation, the aforementioned video materials can be input in the form of static pictures, dynamic pictures (such as gif animations), videos, PPT documents, and PDF documents. It should be understood that each PPT document contains at least one page of slides, and one page of slides is equivalent to a picture. Text, artistic words, pictures, videos and other contents can be inserted into the slides. Similarly, each PDF document contains at least one PDF page, and one PDF page is equivalent to an image, and a PDF page contains elements such as text and images. For example, in an application scenario, a user inputs a PPT document and corresponding dialogue text and video script, and the PPT document contains text, images and videos. This embodiment can generate a video of a virtual person explaining the PPT document based on the input content.

[0031] It should be noted that each slide in the PPT document has corresponding notes, so the dialogue text and video script corresponding to the PPT document can be input as an independent text or as a note of the PPT document. This embodiment does not limit this.

[0032] like Figure 1 Said method specifically comprises the following steps:

[0033] S110, obtaining input content.

[0034] The input content includes a dialogue text and a video script, which is used to generate the dubbing and picture of the video. The dialogue text can be directly input by the user through the client, or obtained from the local storage of the client, or downloaded from the server. The acquisition method of the dialogue text is not limited here. The dialogue text can be a manuscript used for the virtual person to explain the content, and can be expressed in a certain document format, such as: TXT document, WORD document or PDF document. The dialogue text is used to generate the dubbing of the video, that is, the audio of the virtual person. The video script can be understood as the guidance content of the video production, that is, the video script is used to generate the picture of the video. It at least includes the following contents: pause marks corresponding to the dialogue text, display content marks, lens scene marks, lens switching marks, camera movement marks and preset action marks. The video script is pre-edited by the user, that is, the content contained in the video script is designed according to the personalized needs of the user.

[0035] Optionally, after obtaining the input content, the following steps are also included: generating dubbing of the video based on the dialogue text and corresponding pause marks; and using the timeline of the dubbing as the timeline of the video.

[0036] Among them, the pause mark can be used to mark the blank silent period between sentences or between phrases. The pause mark corresponding to the dialogue text can include the position of the pause and the duration of the pause. Based on the dialogue text and the corresponding pause mark, the way to generate the dubbing of the video can be: convert the dialogue text into an audio containing a pause, and use the audio as the dubbing of the virtual person. It should be understood that if the audio is generated only based on the dialogue text, the pause in the audio is mainly determined by the punctuation marks in the dialogue text, and the generated dubbing has a strong "machine sense" and does not conform to the speaking habits of real people. In order to make the generated dubbing more realistic, this embodiment allows the dialogue text to be set with a pause mark. For example, a 0.5 second pause is set at a place where there is no punctuation but a short pause is required, and a 2 second pause is set at a place where there is a period but a longer pause is required. Therefore, on the basis of the prior art of pausing based on punctuation marks, a corresponding pause mark is added to adjust the speech speed and pause of the dubbing, so that the entire dubbing is more realistic. In addition, by adjusting the speech speed and pause, the dubbing can also achieve the effect of emotional expression.

[0037] In this embodiment, after obtaining the dialogue text and its corresponding pause marks, the dialogue text can be converted into the audio containing pauses corresponding to the voice template (such as baritone, bass, soprano, etc.) selected by the user. The voice template can be a pre-designed sound for the user to select or generated by the voice recorded by the user, and the voice template selected by the user matches the image of the virtual person. That is, a suitable timbre is selected for the dubbing, so that the image of the virtual person is consistent in vision and hearing, and the entire character image is richer and fuller.

[0038] It should be noted that the main content of the virtual human video generated by this embodiment is the explanation of the virtual human, so the dubbing generated based on the dialogue text and the corresponding pause marks will run through the entire virtual human video, and the duration of the dubbing is the same as the duration of the virtual human video, and the time axis of the two is also the same. Optionally, the input content can also include an image, which can be a picture or a video, which is reflected in the generated virtual human video in the form of display content to assist the virtual human in explaining the content.

[0039] It is understandable that in some formal occasions, such as product launches, education and training classes, only verbal explanations are boring, and pictures or videos are needed to deepen the audience's understanding of the explanation content and leave a deep impression. Therefore, this embodiment also allows users to input pictures or videos as explanation materials, combined with the language and actions of the virtual person, to form a high-quality video.

[0040] Based on the above description, it can be known that this embodiment can generate the dubbing of the video based on the dialogue text and the corresponding pause marks, and then generate the corresponding virtual human video based on the dubbing. Therefore, one possible implementation method is to directly use the dubbing instead of the dialogue text as the input content, and use the time axis of the dubbing as the time axis of the video.

[0041] Correspondingly, if the input content is a dialogue text, the aforementioned video script can be marked in the dialogue text or on the timeline. If the input content is a dubbing, the aforementioned video script is directly marked on the timeline.

[0042] S120, determining at least one moving time point based on the dialogue text and the video script.

[0043] The moving time point may be a time point on the dubbing time axis, and the moving time point may be the time point when the virtual person switches from a stop state to a moving state (also referred to as a walking state), that is, the starting time point of a certain moving process. Determining multiple moving time points on the dubbing time axis may allow the virtual person to present a stop-and-go state during the process of explaining the content.

[0044] Based on the description of the background technology, it can be known that the related technology can automatically generate a virtual human video based on audio input, but the walking or movement of the virtual human in the video is completely controlled by the video generation software, and the walking or movement of the virtual human cannot be accurately controlled, and it is easy to slip. In order to solve this problem, this embodiment first controls the starting time point of the walking or movement of the virtual human by determining the moving time point.

[0045] Based on the above description of dubbing, it can be known that the entire virtual human video is generated by adding corresponding video images on the basis of dubbing, and the video images specifically include elements such as the picture background and the virtual human image.

[0046] After the dubbing is generated, the corresponding time point and time period of each word in the dialogue text can be clearly found on the time axis. Therefore, the virtual person can be controlled to make corresponding actions at the specified time point and time period, making the virtual person's words and deeds more realistic.

[0047] In daily work and life, when real people are explaining something, they usually walk or move during the intervals between explanations or at regular intervals.

[0048] Correspondingly, in this embodiment, the moving time point is also determined based on the sentence segmentation in the dialogue text or the speaking rhythm in the dubbing or according to a certain frequency.

[0049] Optionally, the method of determining at least one moving time point based on the dialogue text and the video script may be: marking at least one moving time point on the timeline based on the dialogue text and the video script.

[0050] Among them, marking at least one moving time point on the time axis includes at least one of the following methods: marking the time point of a preset type of punctuation mark in the dialogue text on the time axis as a moving time point; or marking the time point of the breath in the dubbing on the time axis as a moving time point; or marking a moving time point on the time axis at set time intervals.

[0051] Among them, the preset type of punctuation mark can be a symbol that can indicate the end of a sentence, such as a period, an exclamation mark, etc. Based on the above description, it can be known that the voice corresponding to each word in the dialogue text can find the corresponding time point and time period on the time axis. Similarly, each punctuation mark in the dialogue text in this embodiment can also find the corresponding time point on the time axis, and then mark the time point corresponding to the preset type of punctuation mark as a moving time point.

[0052] Among them, the breath can also be called a breath, that is, the time point where the breath is located can be understood as the time point where the breath is located, which can be determined according to the speaking rhythm of the dubbing. In this embodiment, the speaking rhythm of the dubbing is analyzed, the time point of the breath on the time axis is obtained, and the time point is determined as the moving time point. It should be noted that the recognition of the aforementioned preset type of punctuation marks is performed on the line text, while the recognition of the breath here is performed on the dubbing, and the pauses and silences in the dubbing are determined as breaths, and then the time points corresponding to the breaths are marked as moving time points.

[0053] The set duration may be a preset value, which is not limited here. Specifically, a moving time point is marked on the time axis every set duration.

[0054] In addition to the several methods of marking moving time points listed above, moving time points can also be marked according to other preset rules, such as marking the time points corresponding to specified words in the dialogue text as moving time points, which is not limited in this embodiment.

[0055] Optionally, after obtaining the input content, the following step is also included: dividing the time axis into a movable period, an immovable period and a movement pending period based on the video script.

[0056] The video script also includes at least one of the following: display content mark, shot angle mark, shot switching mark, camera movement mark and preset action mark. The movable period can be understood as the period during which the virtual person can move, the immovable period can be understood as the period during which the virtual person is prohibited from moving, and the movement pending period can be understood as the period during which the virtual person can move if certain conditions are met; if certain conditions are not met, the virtual person is prohibited from moving. In this application scenario, in the process of the virtual person introducing the content, the virtual person does not need to move or cannot move under some preset circumstances, so the timeline needs to be divided into movable period, immovable period and movement pending period.

[0057] Among them, the display content mark is used to mark the display method of the content within a certain period of time, such as full-screen display or non-full-screen display; the lens view mark is used to mark the lens view of a certain period of time, such as: panoramic view, long shot, medium shot, close-up and close-up, etc.; the lens switching mark is used to mark the switching of the lens at a certain moment; the camera movement mark is used to mark the camera movement method of a certain period of time, for example: push shot, pull shot, lifting shot and shaking shot, etc.; the preset action mark is used to mark the pre-designed action performed by the virtual person in a certain period of time.

[0058] Specifically, the method of dividing the time axis into movable time periods, non-movable time periods and moving pending time periods based on the video script may be: determining the time periods marked as full-screen display, the time periods corresponding to the lens switching marks and the time periods corresponding to the preset action marks as non-movable time periods; determining the time periods marked as panoramic view as moving pending time periods; and determining the remaining time periods on the time axis as movable time periods. For example, assuming that Figure 2 This is an example diagram of dividing the time axis into a movable period, an immovable period, and a movable pending period in this embodiment, as shown in FIG. Figure 2 As shown, the overall length of the time axis is 300 seconds, which is divided into two non-movable periods, namely 50 seconds to 80 seconds and 200 seconds to 259 seconds, one pending movement period, namely 90 seconds to 100 seconds, and the rest of the periods are movable periods.

[0059] In this embodiment, the time axis is divided into a movable period, an immovable period and a movement pending period, so that the virtual person cannot move in the immovable period, thereby strengthening the control over the walking or movement of the virtual person.

[0060] Specifically, during the full-screen display period, the entire video screen will display the explanation material (picture or video) input by the user in full screen, and the virtual person image will no longer be displayed. At this time, the virtual person does not need to walk or move. During the period corresponding to the lens switching mark, the video screen will switch lenses. If the virtual person walks or moves during this period, it may cause the position of the virtual person before and after the lens switching to shift instantly. During the period corresponding to the preset action mark, the upper body of the virtual person needs to make preset actions, such as waving, making a heart shape, folding hands, raising hands, etc. If the virtual person walks or moves while doing these actions, the body movements will be uncoordinated. Therefore, this embodiment classifies the aforementioned periods as immovable periods, and prohibits the virtual person from walking or moving during the immovable periods.

[0061] During the panoramic lens period, the video screen fully displays the virtual person and the screen background from a distance. The panoramic lens mainly shows the audience the entire video screen, rather than focusing on the virtual person. If the virtual person walks or moves for a long time during this period, the audience will focus on the virtual person and ignore other elements in the video screen, such as the explanation materials (pictures or videos) input by the user. Therefore, this embodiment classifies the panoramic lens period as a movement pending period, allowing the virtual person to walk or move briefly during the movement pending period, but not for a long time.

[0062] S130: Add at least one lower body movement animation at each movement time point to obtain an animation set corresponding to each movement time point.

[0063] Wherein, the moving time point is the starting time point of the lower body moving animation playback, that is, the added lower body moving animation starts to play from the moving time point. The lower body moving animation can be a pre-made animation of the lower body movement of the virtual person or the lower body walking, and is stored in the material library. The lower body moving animation includes information such as the starting position, target position and moving duration of the movement. The difference between a variety of lower body moving animations can be that the starting position and / or target position of the movement are different. Exemplary, assuming that the lower body moving animation produced is an animation of the movement of the virtual person between three position points a, b and c, the lower body moving animation included can include: a→b (5 seconds), a→c (7 seconds), b→a (10 seconds), b→c (8 seconds), c→a (7 seconds) and c→b (9 seconds).

[0064] It should be noted that in the related art, the walking or moving action of the virtual person is directly generated by the video generation software based on the audio input, while in addition to controlling the starting time point of the virtual person's walking or moving by determining the moving time point, this embodiment also prevents the occurrence of sliding by pre-making the lower body moving animation. It should be understood that this embodiment pre-makes a variety of lower body moving animations and stores them in the material library. When generating the virtual person video, only a certain lower body moving animation can be selected from the material library as the movement of the virtual person's lower body, and the starting position, target position and moving duration of the movement are deterministic. In addition, these pre-made lower body moving animations are carefully designed and repeatedly tested by designers, and the model quality is high and the robustness is good.

[0065] That is to say, the related technology directly generates the walking or moving movements of the virtual human based on audio input, while this embodiment selects a pre-made lower body movement animation in the material library as the lower body movement of the virtual human walking or moving, which can effectively prevent the occurrence of sliding.

[0066] In this embodiment, when adding the lower body movement animation to the moving time point, it can be determined whether to add the lower body movement animation to the moving time point based on the overlap between the lower body movement animation and the immovable period or the movement pending period.

[0067] Specifically, at least one lower body movement animation is added to each movement time point, and the method for obtaining the animation set corresponding to each movement time point can be: obtaining the movement duration of at least one lower body movement animation; determining the movement time period corresponding to the lower body movement animation on the time axis according to the movement duration and the movement time point; determining the overlapping results of the movement time period corresponding to each lower body movement animation with the immovable time period and the movement pending time period; adding at least one lower body movement animation to each movement time point based on the overlapping results, to obtain the animation set corresponding to each movement time point.

[0068] It should be understood that each lower body movement animation will last for a period of time, which corresponds to a movement duration. Therefore, in order to determine whether adding a certain lower body movement animation at a certain movement time point will cause the virtual person to move during the immovable period and the movement pending period, it is necessary to mark the duration of the lower body movement animation on the timeline.

[0069] Among them, according to the moving duration and the moving time point, the method of determining the corresponding moving period of the lower body moving animation on the time axis can be: determining the moving time point as the playback starting point of the lower body moving animation on the time axis, accumulating the moving time point and the moving duration, obtaining the playback end point of the lower body moving animation on the time axis, and thus obtaining the corresponding moving period of the lower body moving animation on the time axis. Exemplarily, assuming that the moving time point is the 50th second, and the moving duration of a lower body moving animation is 10 seconds, then the corresponding moving period on the time axis of the lower body moving animation is the 50th second to the 60th second.

[0070] Wherein, the overlapping results include overlapping and non-overlapping. In the present embodiment, if the moving period corresponding to the lower body moving animation and the immovable period have an intersection, the two overlap; if the moving period corresponding to the lower body moving animation and the immovable period do not have an intersection, the two do not overlap. Similarly, if the moving period corresponding to the lower body moving animation and the moving pending period have an intersection, the two overlap; if the moving period corresponding to the lower body moving animation and the moving pending period do not have an intersection, the two do not overlap. In the present embodiment, based on the overlapping results of the moving period corresponding to the lower body moving animation and the immovable period and the moving pending period, it is determined whether the lower body animation is added to the moving time point. Exemplary, assuming that the corresponding moving period on the time axis of the lower body moving animation is the 50th second to the 60th second, and the immovable period is the 57th second to the 80th second, then the two periods overlap, and the overlapping period is the 57th second to the 60th second, with a total overlap of 3 seconds.

[0071] Specifically, the method of adding at least one lower body movement animation to each moving time point based on the overlapping results can be: adding the lower body movement animation that meets the following conditions to the animation set corresponding to the moving time point: the moving period does not overlap with the immovable period and the movement pending period; or, the moving period overlaps with the movement pending period and the overlapping duration is less than the set threshold.

[0072] Wherein, the set threshold value can be pre-set, for example, set to any value between 5 and 10 seconds. In this embodiment, if the moving period of the lower body moving animation on the time axis does not overlap with the immovable period and the moving pending period, the lower body moving animation is added to the animation set corresponding to the moving time point; if the moving period of the lower body moving animation on the time axis overlaps with the immovable period, the lower body moving animation is filtered out, that is, the lower body moving animation is not added to the animation set corresponding to the moving time point. If the moving period of the lower body moving animation on the time axis overlaps with the moving pending period, the overlapping duration is obtained, and if the overlapping duration is less than the set threshold, the lower body moving animation is added to the animation set corresponding to the moving time point; if the overlapping duration is greater than or equal to the set threshold, the lower body moving animation is filtered out, that is, the lower body moving animation is not added to the animation set corresponding to the moving time point.

[0073] Based on the above description, as long as the moving period overlaps with the immovable period, it means that the movement is not allowed, that is, the movement cannot be performed at the moving time point. If the moving period overlaps with the moving pending period, it is necessary to further determine whether the movement is allowed according to the overlapping duration.

[0074] If the animation set corresponding to a certain moving time point is an empty set, it means that there is no suitable lower body movement at the moving time point, and the moving time point is deleted.

[0075] For example, Figure 3 This is an example diagram of adding lower body movement animation at the moving time point in this embodiment. Figure 3 As shown, the overall duration of the time axis is 300 seconds, which includes two immovable time periods, which are 50 seconds to 80 seconds and 200 seconds to 259 seconds, and one mobile pending time period, which is 90 seconds to 100 seconds, and the remaining time periods are all movable time periods. Determine 5 moving time points on the time axis, and find all lower body moving animations that meet the current screen scene and screen style in the preset material library, so as to determine the moving time periods corresponding to these lower body moving animations in turn at the 5 moving time points. Specifically, it is only necessary to use the moving time point as the playback starting point of the lower body moving animation on the time axis, and then add the moving time point and the moving duration of each lower body moving animation, so as to determine the moving time period corresponding to each lower body moving animation at the moving time point on the time axis. Repeat the above operation at each moving time point, and the moving time period corresponding to each lower body moving animation at the 5 moving time points can be determined on the time axis. Figure 3None of the five moving periods overlaps with the immovable period, and the second moving period overlaps with the pending moving period, but the overlapping duration is less than the preset threshold. Then, the lower body moving animations corresponding to the five moving periods are added to the animation set corresponding to each moving time point.

[0076] S140: Determine a target lower body movement animation sequence based on an animation set corresponding to each movement time point.

[0077] The target lower body movement animation sequence is a sequence consisting of selecting a lower body movement animation from the animation set at each movement time point, and is the animation sequence ultimately used to generate the virtual human movement video. In this embodiment, since the animation set at each movement time point includes one or more lower body movement animations, multiple lower body movement animation sequences can be formed, and finally a target lower body movement animation sequence is determined from the multiple lower body movement animation sequences.

[0078] Optionally, the method of determining the target lower body movement animation sequence based on the animation set corresponding to each movement time point can be: based on movement continuity, select a lower body movement animation from the animation set corresponding to each movement time point for combination to obtain at least one candidate lower body movement animation sequence; determine the target lower body movement animation sequence from at least one candidate lower body movement animation sequence.

[0079] Among them, the movement continuity means that the target position of the previous movement animation in two adjacent movement animations is the same as the starting position of the next movement animation. Specifically, the process of selecting a lower body movement animation for combination from the animation set corresponding to each movement time point based on the movement continuity can be: traversing each lower body movement animation in the animation set corresponding to the first movement time point, obtaining the target position of the traversed lower body movement animation, and then selecting a lower body movement animation whose starting position is the same as the target position of the lower body movement animation traversed in the animation set corresponding to the first movement time point from the animation set corresponding to the second movement time point, and then selecting a lower body movement animation whose starting position is the same as the target position of the lower body movement animation selected in the animation set corresponding to the second movement time point from the animation set corresponding to the third movement time point, and so on, until a lower body movement animation is selected in the animation set corresponding to the last movement time point to form a candidate lower body movement animation sequence; when the traversal of each lower body movement animation in the animation set corresponding to the first movement time point is completed, multiple candidate lower body movement animation sequences are obtained.

[0080] In this embodiment, the method of determining the target lower body movement animation sequence from at least one candidate lower body movement animation sequence can be: randomly selecting a candidate lower body movement animation sequence to determine the target lower body movement animation sequence; or, in order to improve the quality of the generated video, evaluating at least one candidate lower body movement animation sequence separately, and determining a target lower body movement animation sequence based on the evaluation results.

[0081] Optionally, a method for determining a target lower body movement animation sequence from at least one candidate lower body movement animation sequence may be: evaluating at least one candidate lower body movement animation sequence based on a set evaluation index to obtain a score for each candidate lower body movement animation sequence; and determining the candidate lower body movement animation sequence with the highest score as the target lower body movement animation sequence.

[0082] The setting of the evaluation index includes at least one of the following: the first movement time, the time interval between two adjacent movements, the movement frequency and the end point position of the movement.

[0083] Among them, the first movement time can be the starting time point of the first lower body movement animation. Specifically, the method of evaluating at least one candidate lower body movement animation sequence based on the first movement time can be: for each candidate lower body movement animation sequence, obtain the deviation between the first movement time of the lower body movement animation sequence and the benchmark first movement time, and obtain the score corresponding to the first movement time based on the deviation. Among them, the benchmark first movement time can be pre-set, and the score of the first movement time of the candidate lower body movement animation sequence is negatively correlated with the deviation, that is, the larger the deviation, the smaller the score, and the smaller the deviation, the higher the score. In this embodiment, the specific relationship between the score and the deviation of the first movement time is not limited.

[0084] Wherein, the time interval between two adjacent movements can be understood as the interval between the end moment of the previous lower body movement animation and the start moment of the next lower body movement animation in two adjacent lower body movement animations. Specifically, the method of evaluating at least one candidate lower body movement animation sequence based on the time interval between two adjacent movements can be: for each candidate lower body movement animation sequence, the time interval between two adjacent lower body movement animations is obtained, and then the deviation between each time interval and the reference time interval is determined respectively, and the score of each time interval is determined based on the deviation, and finally the average value of the score is used as the score corresponding to the time interval between two adjacent movements of the candidate lower body movement animation sequence. Wherein, the reference time interval can be pre-set, and the score of the time interval between adjacent lower body movement animations is negatively correlated with the deviation, that is, the larger the deviation, the smaller the score, and the smaller the deviation, the higher the score. In this embodiment, the specific relationship between the score and the deviation of the time interval between two adjacent movements is not limited.

[0085] Wherein, the movement frequency can be understood as the number of movements per unit time. Specifically, the process of evaluating at least one candidate lower body movement animation sequence based on the movement frequency can be: for each candidate lower body movement animation sequence, the number of movements is divided by the total duration of dubbing to obtain the movement frequency of the candidate lower body movement animation sequence, and then the deviation between the movement frequency and the reference movement frequency is determined, and the score of the movement frequency of the candidate lower body movement animation sequence is determined based on the deviation. Wherein, the reference movement frequency is pre-set, and the score of the movement frequency of the candidate lower body movement animation sequence is negatively correlated with the deviation, that is, the larger the deviation, the smaller the score, and the smaller the deviation, the higher the score. In this embodiment, the specific relationship between the score and the deviation of the movement frequency is not limited.

[0086] Among them, the end position can be understood as the position where the virtual person finally stops, or the target position of the last lower body moving animation in the candidate lower body moving animation sequence. Specifically, the method of evaluating at least one candidate lower body moving animation sequence based on the end position can be: for each candidate lower body moving animation sequence, obtain the deviation between the end position of the lower body moving animation sequence and the reference end position time, and obtain the score corresponding to the end position based on the deviation. Among them, the reference end position can be pre-set, and the score of the end position of the candidate lower body moving animation sequence is negatively correlated with the deviation, that is, the larger the deviation, the smaller the score, and the smaller the deviation, the higher the score. In this embodiment, the specific relationship between the score and the deviation of the end position is not limited.

[0087] In this embodiment, after obtaining the scores corresponding to multiple set evaluation indicators, the scores corresponding to the multiple set evaluation indicators are weighted and summed to obtain the scores of each candidate lower body movement animation sequence, and finally the candidate lower body movement animation sequence with the highest score is determined as the target lower body movement animation sequence.

[0088] S150, generating a virtual human full body movement video according to the target lower body movement animation sequence and the input content.

[0089] The virtual human full body movement video is a full body video including the virtual human upper body movement animation and lower body movement animation.

[0090] Specifically, the process of generating a virtual human full body movement video according to a target lower body movement animation sequence and input content may be: generating a virtual human lower body movement video according to a target lower body movement animation sequence; generating a virtual human full body movement video according to the virtual human lower body movement video and input content.

[0091] Among them, the process of generating a virtual human lower body moving video according to the target lower body moving animation sequence can be: each lower body moving animation in the target lower body moving animation sequence is connected in sequence, and in the connection process, the video clip between two adjacent lower body moving animations is filled with a plurality of pre-designed lower body standing animations. Specifically, there are slight differences between different lower body standing animations, and the duration of each lower body standing animation is also different. When filling the video clip, a lower body standing animation is randomly selected from them and connected in series with the previous lower body moving animation. If the video clip is not filled, a lower body standing animation is randomly selected again and connected in series with the previous lower body standing animation until the video clip is filled. If the duration of the connected lower body standing animation exceeds the duration of the video clip, the lower body standing animation can also be cut so that it just fills the video clip, thereby achieving the filling of video clips of any duration, thereby connecting two adjacent lower body moving animations in series. That is, in the generated virtual human lower body movement video, between two lower body movement animations, the virtual human stops at the target position of the previous lower body movement animation and keeps standing in the same position until the next lower body movement animation starts.

[0092] In this embodiment, the method of generating a full-body movement video of a virtual person based on the lower-body movement video of the virtual person and the input content can be: inputting the lower-body movement video of the virtual person and the input content into a pre-trained first generation model, and outputting the full-body movement video of the virtual person.

[0093] Optionally, the method of generating a full-body moving video of the virtual person based on the lower-body moving video of the virtual person and the input content may be: generating a full-body moving video of the virtual person based on the lower-body moving video of the virtual person and the input content; and fusing the lower-body moving video of the virtual person and the upper-body moving video of the virtual person to obtain a full-body moving video of the virtual person.

[0094] The video frames of the upper body movement video of the virtual person may correspond to the video frames of the lower body movement video of the virtual person one by one. In this embodiment, the method of generating the upper body movement video of the virtual person according to the lower body movement video of the virtual person and the input content may be: inputting the lower body movement video of the virtual person and the dubbing of the video into a pre-trained second generation model, and outputting the upper body movement video of the virtual person.

[0095] In this embodiment, the method of fusing the virtual person's lower body moving video and the virtual person's upper body moving video can be: input the virtual person's lower body moving video and the virtual person's upper body moving video into a pre-trained fusion model to output the virtual person's full body moving video. Alternatively, the virtual person's lower body moving video and the virtual person's upper body moving video are decoded separately to obtain multiple lower body video frames and multiple upper body video frames, and then the lower body video frames are fused with the corresponding upper body video frames to obtain full body video frames; finally, the full body video frames are encoded to obtain the virtual person's full body moving video.

[0096] Figure 4 is a structural diagram of a virtual human video generation device provided by an embodiment of the present invention, such as Figure 4 As shown, the device comprises:

[0097] Input content acquisition module 410, used to acquire input content; wherein the input content includes dialogue text and video script, which is used to generate the dubbing and picture of the video;

[0098] A moving time point determination module 420, for determining at least one moving time point based on the dialogue text and the video script;

[0099] The lower body movement animation adding module 430 is used to add at least one lower body movement animation at each movement time point to obtain an animation set corresponding to each movement time point; wherein the movement time point is the starting time point of the lower body movement animation;

[0100] A target lower body movement animation sequence determination module 440 is used to determine a target lower body movement animation sequence based on an animation set corresponding to each movement time point;

[0101] The virtual human full body movement video generation module 450 is used to generate a virtual human full body movement video according to the target lower body movement animation sequence and input content.

[0102] Optionally, the video script includes pause marks corresponding to the dialogue text, and also includes: a video timeline acquisition module, which is used to:

[0103] Generate video dubbing based on the dialogue text and corresponding pause markers;

[0104] Use the timeline of the dubbing as the timeline of the video.

[0105] Optionally, the moving time point determination module 420 is further configured to:

[0106] Marking at least one moving time point on the timeline based on the dialogue text and the video script;

[0107] The marking of at least one moving time point on the time axis includes at least one of the following methods:

[0108] Mark the time points of the preset punctuation marks in the dialogue text on the timeline as moving time points; or,

[0109] Mark the time points of the breath in the dubbing on the timeline as moving time points; or,

[0110] Mark a moving time point on the timeline at set intervals.

[0111] Optionally, it also includes: a time period division module, which is used to:

[0112] The timeline is divided into movable time periods, immovable time periods and movable pending time periods based on the video script; wherein the video script also includes at least one of the following: display content mark, lens scene mark, lens switching mark, camera movement mark and preset action mark.

[0113] Optionally, the lower body movement animation adding module 430 is further used for:

[0114] Get the moving duration of at least one lower body moving animation;

[0115] According to the movement duration and movement time point, determine the movement period corresponding to the lower body movement animation on the timeline;

[0116] Determine the overlapping results of the moving time period, the immovable time period and the moving pending time period corresponding to each lower body moving animation;

[0117] At least one lower body movement animation is added to each movement time point based on the overlapping result to obtain an animation set corresponding to each movement time point.

[0118] Optionally, the lower body movement animation adding module 430 is further used for:

[0119] The lower body movement animation that meets the following conditions is added to the animation set corresponding to the movement time point: the movement period does not overlap with the immovable period and the movement pending period; or the movement period overlaps with the movement pending period and the overlapping duration is less than the set threshold.

[0120] Optionally, the target lower body movement animation sequence determination module 440 is further configured to:

[0121] Based on the movement continuity, a lower body movement animation is selected from the animation set corresponding to each movement time point for combination to obtain at least one candidate lower body movement animation sequence; wherein the movement continuity is that the target position of the previous movement animation is the same as the starting position of the next movement animation in two adjacent movement animations;

[0122] A target lower body movement animation sequence is determined from at least one candidate lower body movement animation sequence.

[0123] Optionally, the target lower body movement animation sequence determination module 440 is further configured to:

[0124] Evaluate at least one candidate lower body movement animation sequence based on a set evaluation index to obtain a score for each candidate lower body movement animation sequence; wherein the set evaluation index includes at least one of the following: first movement time, time interval between two adjacent movements, movement frequency, and end point position of movement;

[0125] The candidate lower body movement animation sequence with the highest score is determined as the target lower body movement animation sequence.

[0126] Optionally, the virtual human full body moving video generation module 450 is further used for:

[0127] Generate a virtual human lower body movement video according to the target lower body movement animation sequence;

[0128] A full-body movement video of the virtual person is generated according to the lower-body movement video of the virtual person and input content.

[0129] Optionally, the virtual human full body moving video generation module 450 is further used for:

[0130] Generate a virtual human upper body movement video according to the virtual human lower body movement video and input content;

[0131] The virtual human lower body movement video and the virtual human upper body movement video are fused to obtain the virtual human full body movement video.

[0132] The above device can execute the methods provided by all the above embodiments of the present invention, and has the corresponding functional modules and beneficial effects of executing the above methods. For technical details not described in detail in this embodiment, please refer to the methods provided by all the above embodiments of the present invention.

[0133] Figure 5 A schematic diagram of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, the connections and relationships between the components, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0134] like Figure 5As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0135] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0136] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as a method for generating a virtual human video.

[0137] In some embodiments, the method for generating a virtual human video may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for generating a virtual human video described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the method for generating a virtual human video in any other appropriate manner (e.g., by means of firmware).

[0138] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0139] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0140] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0142] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0143] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0144] An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the method for generating a virtual human video as provided in any embodiment of the present application.

[0145] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0147] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for generating a virtual human video, characterized in that: include: Obtaining input content; wherein the input content includes dialogue text and video script, which is used to generate the dubbing and picture of the video; Determining at least one moving time point based on the dialogue text and the video script; Add at least one lower body movement animation at each movement time point to obtain an animation set corresponding to each movement time point; wherein the movement time point is the starting time point of the lower body movement animation, and the starting position, target position and movement duration of the lower body movement animation are deterministic; Determine the target lower body movement animation sequence based on the animation set corresponding to each of the movement time points; A virtual human full-body movement video is generated according to the target lower body movement animation sequence and the input content.

2. The method according to claim 1, characterized in that The video script includes pause marks corresponding to the dialogue text, and after obtaining the input content, further includes: Generating a dubbing of the video based on the dialogue text and corresponding pause marks; The timeline of the dubbing is used as the timeline of the video.

3. The method according to claim 2, characterized in that Determining at least one moving time point based on the dialogue text and the video script includes: Marking at least one of the moving time points on the timeline based on the dialogue text and the video script; Wherein, marking at least one of the moving time points on the time axis includes at least one of the following methods: Marking a time point of a punctuation mark of a preset type in the dialogue text on the time axis as the moving time point; or, Marking the time point of the breath in the dubbing on the time axis as the moving time point; or, A moving time point is marked on the time axis at intervals of a set time.

4. The method according to claim 2, characterized in that: After getting the input content, it also includes: Based on the video script, the timeline is divided into a movable period, an immovable period and a movable pending period; wherein the video script also includes at least one of the following: display content mark, shot view mark, shot switching mark, camera movement mark and preset action mark.

5. The method according to claim 4, characterized in that Adding at least one lower body movement animation at each movement time point to obtain an animation set corresponding to each movement time point includes: Get the moving duration of at least one lower body moving animation; Determine, according to the movement duration and the movement time point, a movement period corresponding to the lower body movement animation on the time axis; Determine the overlapping result of the moving time period corresponding to each of the lower body moving animations and the immovable time period and the moving pending time period; At least one lower body movement animation is added to each movement time point based on the overlapping result to obtain an animation set corresponding to each movement time point.

6. The method according to claim 5, characterized in that Adding at least one lower body movement animation to each movement time point based on the overlapping result, including: The lower body movement animation that meets the following conditions is added to the animation set corresponding to the movement time point: the movement period does not overlap with the immovable period and the movement pending period; or, the movement period overlaps with the movement pending period and the overlapping duration is less than a set threshold.

7. The method according to claim 1, characterized in that Determining the target lower body movement animation sequence based on the animation set corresponding to each of the movement time points includes: Based on the movement continuity, a lower body movement animation is selected from the animation set corresponding to each movement time point for combination to obtain at least one candidate lower body movement animation sequence; wherein the movement continuity is that the target position of the previous movement animation is the same as the starting position of the next movement animation in two adjacent movement animations; A target lower body movement animation sequence is determined from the at least one candidate lower body movement animation sequence.

8. The method according to claim 7, characterized in that Determining a target lower body movement animation sequence from the at least one candidate lower body movement animation sequence comprises: The at least one candidate lower body movement animation sequence is evaluated based on a set evaluation index to obtain a score for each candidate lower body movement animation sequence; wherein the set evaluation index includes at least one of the following: first movement time, time interval between two adjacent movements, movement frequency, and end point position of movement; The candidate lower body movement animation sequence with the highest score is determined as the target lower body movement animation sequence.

9. The method according to claim 1, characterized in that: Generating a virtual human full body movement video according to the target lower body movement animation sequence and the input content, including: Generate a virtual human lower body movement video according to the target lower body movement animation sequence; A full-body movement video of the virtual person is generated according to the lower-body movement video of the virtual person and the input content.

10. The method according to claim 9, characterized in that Generating a full-body movement video of a virtual person according to the lower-body movement video of the virtual person and the input content includes: Generate a virtual human upper body movement video according to the virtual human lower body movement video and the input content; The virtual human lower body movement video and the virtual human upper body movement video are fused to obtain the virtual human full body movement video.

11. A virtual human video generation device, characterized in that: include: An input content acquisition module, used to acquire input content; wherein the input content includes dialogue text and video script, which is used to generate the dubbing and picture of the video; A moving time point determination module, used to determine at least one moving time point based on the dialogue text and the video script; A lower body movement animation adding module, used to add at least one lower body movement animation at each movement time point, and obtain an animation set corresponding to each movement time point; wherein the movement time point is the starting time point of the lower body movement animation, and the starting position, target position and movement duration of the lower body movement animation are deterministic; A target lower body movement animation sequence determination module, used to determine the target lower body movement animation sequence based on the animation set corresponding to each movement time point; The virtual human full body movement video generation module is used to generate a virtual human full body movement video according to the target lower body movement animation sequence and the input content.

12. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory in communication with the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for generating a virtual human video according to any one of claims 1 to 10.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the method for generating a virtual human video according to any one of claims 1 to 10 when executed.

14. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the computer program implements the method for generating a virtual human video according to any one of claims 1 to 10.

Images