Video generation method and device, readable medium, electronic equipment and program product

Abstract

The invention discloses a video generation method and device, a readable medium, electronic equipment and a program product. The method comprises the following steps: determining a target text for video generation; determining a plurality of split documents according to the target text, and generating a plurality of split pictures for each split document; according to a preset grouping strategy, the split picture generated by each split document is divided into a plurality of picture groups, one picture group comprises one split picture generated by each split document, and the split pictures corresponding to the same split document in different picture groups are different; and generating a target video at least according to the split picture in at least one picture group in the plurality of picture groups. Therefore, the text content of the target text can be quickly converted into the video content through an automatic process, so that the video production difficulty is reduced, and the video production efficiency is improved. In addition, when the video is generated based on the picture group, diversified video content selection can be provided for the user, and the video quality is prevented from being influenced by homogenization of the split pictures.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular, to a video generation method, device, readable medium, electronic device, and program product. Background Art

[0002] With the development of computer technology, information can be delivered to users based on different content delivery methods. For example, information can be delivered to users in the form of videos.

[0003] However, in the related art, when delivering information to users via video, it is necessary to first produce a video based on the video material, which has the problems of great difficulty in video production and low efficiency in video production. Summary of the invention

[0004] This summary is provided to introduce concepts in a brief form that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0005] In a first aspect, the present disclosure provides a video generation method, the video generation method comprising: Determine the target text for video generation; Determine a plurality of storyboards according to the target text, and generate a plurality of storyboard images for each of the storyboards; According to a preset grouping strategy, the storyboard pictures generated by each storyboard copy are divided into a plurality of picture groups, wherein one picture group includes one storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different; A target video is generated based on at least one storyboard picture in at least one of the plurality of picture groups.

[0006] In a second aspect, the present disclosure provides a video generation device, the video generation device comprising: A determination module, used for determining a target text for video generation; A first processing module is used to determine a plurality of storyboards according to the target text, and generate a plurality of storyboard images for each of the storyboards; A second processing module is used to divide the storyboard pictures generated by each storyboard copy into a plurality of picture groups according to a preset grouping strategy, wherein one picture group includes one storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different; The third processing module is used to generate a target video based on at least the storyboard pictures in at least one picture group among the plurality of picture groups.

[0007] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in the first aspect when the computer program is executed by a processing device.

[0008] In a fourth aspect, the present disclosure provides an electronic device, including: a storage device having a computer program stored thereon; A processing device is used to execute the computer program in the storage device to implement the steps of the method in the first aspect.

[0009] In a fifth aspect, the present disclosure provides a computer program product, comprising a computer program, which implements the steps of the method described in the first aspect when executed by a processor.

[0010] Through the above technical scheme, multiple storyboards can be determined according to the target text, and multiple storyboard pictures can be generated for each storyboard, and the storyboard pictures can be divided into multiple picture groups according to a preset grouping strategy, and at least the target video can be generated according to the storyboard pictures in at least one picture group in the multiple picture groups. Thus, the text content of the target text can be quickly converted into video content through an automated process, thereby reducing the difficulty and operation complexity of video production and improving video production efficiency. In addition, since a picture group can include a storyboard picture generated by each storyboard, and the storyboard pictures corresponding to the same storyboard in different picture groups are different. Thus, when generating a video based on the storyboard pictures in the picture group, the coherence of the video content can be guaranteed, and a variety of video content selections can be provided for users, avoiding the influence of the video quality due to the homogenization of the storyboard pictures.

[0011] Other features and advantages of the present disclosure will be described in detail in the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale. In the drawings: Figure 1 is a flowchart of a video generation method according to an exemplary embodiment of the present disclosure; Figure 2 is a flowchart of another video generation method according to an exemplary embodiment of the present disclosure; Figure 3is a structural block diagram of a video generating device according to an exemplary embodiment of the present disclosure; Figure 4 It is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0013] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0014] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0015] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0016] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0017] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0018] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0019] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0020] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.

[0021] As an optional but non-limiting implementation, in response to receiving an active request from the user, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. In addition, the pop-up window may also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0022] It is understandable that the above notification and the process of obtaining user authorization are merely illustrative and do not constitute a limitation on the implementation of the present disclosure. Other methods that meet the relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0023] At the same time, it is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and relevant provisions.

[0024] As mentioned in the background technology, when the related technology transmits information to users through video, it is necessary to first produce a video based on the video material, which has problems such as great difficulty in video production and low efficiency in video production.

[0025] For example, in the field of content push, when pushing advertisements to users via videos, stories or plot clips are generally added to the advertising creative copy, so that when making videos based on the advertising creative copy, the video content can be enriched or the video can be made more visual and story-like, thereby improving the conversion effect of the advertisement.

[0026] However, when producing videos based on advertising creative copy, related technologies can generally produce videos in the following two ways: Method 1: Cross-platform video production. First, the creative advertising copy is imported into platform 1, and the creative advertising copy is split into multiple storyboards through platform 1; secondly, the storyboards are imported into platform 2, and a picture material is generated for each storyboard through platform 2; then the picture material is imported into platform 3, and all the picture materials are edited into a video through platform 3; finally, dubbing is performed through platform 3 to obtain the target audio, and the target audio is imported into platform 3, and the target audio and video are synthesized through platform 3 to obtain the edited video.

[0027] Method 2: Use a one-stop tool to make a video. This means importing the creative copy into the one-stop tool, manually breaking down the creative copy into a storyboard copy, and then manually writing the prompt for generating a storyboard image based on the storyboard copy in the one-stop tool. Finally, generating a storyboard image based on the prompt, and rendering the storyboard image into a video.

[0028] In method 1, since video production needs to be carried out across multiple platforms, for video producers without video production experience, there are problems such as difficulty in video production and low efficiency. In method 2, since the storyboards need to be split manually and prompts need to be written manually, there are problems such as low splitting efficiency and low writing efficiency, which in turn affects the efficiency of video production. In addition, when writing prompts manually, there are situations where prompts written by different storyboards are the same or similar, which leads to the problem of homogenization of the picture materials, which in turn affects the video quality.

[0029] In view of this, the present disclosure provides a video generation method, device, readable medium, electronic device and program product to solve the above technical problems.

[0030] It is worth noting that the video generation method in the present disclosure can be used for video generation in different fields, such as novel fields, game fields, financial fields or social fields, and the embodiments of the present disclosure do not impose any restrictions on this.

[0031] The embodiments of the present disclosure are further explained below with reference to the accompanying drawings.

[0032] Figure 1 is a flowchart of a video generation method according to an exemplary embodiment of the present disclosure, referring to Figure 1 , the method may include the following steps: S101: Determine a target text for video generation.

[0033] In this embodiment, the target text can be text manually entered by the user, can be text selected from a script library, can be text extracted from audio, pictures or videos, and of course can be text obtained by other means, and the present embodiment does not impose any restrictions on this.

[0034] For example, when a user opens a video production application in a terminal device, a function interface of the video production application can be displayed in the terminal device, and the function interface can display a script generation control "story creation" for generating a video script. After the script generation control "story creation" is triggered by operations such as clicking, a script editing page is displayed, and the script editing page can display a script input box, a script extraction control, and a script library control. When the user triggers the script input box, the user can enter text by voice or natural language text to obtain a target text for video generation. When the user triggers the script extraction control by operations such as clicking, an upload page for uploading images, audio, or video can be displayed. After uploading images, audio, or video on the upload page, the corresponding text can be automatically extracted from the image, audio, or video, and can be displayed in the script input box, so that the user can adjust the extracted text, and then obtain the target text for video generation. When the user triggers the script library control by operations such as clicking, a preset video script and / or a historically generated video script can be displayed. By clicking the target video script, the target video script can be displayed in the text input box, so that the user can adjust the target video script, and then obtain the target text for video generation.

[0035] S102: Determine a plurality of storyboards according to the target text, and generate a plurality of storyboard images for each storyboard.

[0036] In this embodiment, the multiple storyboards can be obtained by splitting the target text based on punctuation marks, or by splitting the target text based on punctuation marks and a preset storyboard length threshold, or by other methods, which are not limited in this embodiment. The punctuation marks can be a period, a line break, or a section break, etc.

[0037] For example, Figure 2 As shown, if the target text is determined as follows: Not long after the June college entrance examination, Guan Xuxu went home alone after completing the discharge procedures. As soon as she stepped into the gate of Guan's villa, a suitcase fell at her feet with a "clang". The beautiful woman dressed elegantly stood at the entrance and looked down at her. Her eyes swept across the girl's delicate features and snow-white skin. A trace of jealousy flashed in her eyes, followed by undisguised disgust. The target text can be split based on the period, and the following 3 storyboards are obtained: Storyboard 1: Shortly after the college entrance examination in June, Guan Xuxu completed the discharge procedures and returned home alone.

[0038] Storyboard 2: As soon as she stepped into the door of Guan’s villa, a suitcase crashed at her feet with a bang.

[0039] Storyboard copy 3: The beautiful woman dressed elegantly stood at the entrance and looked down at her, her eyes sweeping across the girl's delicate features and snow-white skin. A hint of jealousy flashed in her eyes, followed by undisguised disgust.

[0040] For example, the target text can be split based on the period and the preset storyboard copy length threshold. If the determined target text is: Not long after the June college entrance examination, Guan Xuxu went home alone after completing the discharge procedures. As soon as she stepped into the gate of Guan's villa, a suitcase fell at her feet with a "clang". The beautiful woman dressed in noble clothes stood at the entrance and looked down at her. Her eyes swept across the girl's delicate features and snow-like skin. A trace of jealousy flashed in her eyes, followed by undisguised disgust. If the preset storyboard copy length threshold is 45 words, the target text can be split based on punctuation marks, such as periods, to obtain the above three storyboard copies. However, since the length of storyboard copy 3 is greater than the preset storyboard copy length threshold of 45 words, storyboard copy 3 can be further split based on other punctuation marks in storyboard copy 3. For example, storyboard copy 3 can be split into storyboard copy "A beautiful woman dressed elegantly stood at the entrance and looked down at her" and storyboard copy "His eyes swept across the girl's delicate features and snow-white skin, a trace of jealousy flashed in his eyes, followed by undisguised disgust", thus, the following four storyboard copies can be obtained: Storyboard 1: Shortly after the college entrance examination in June, Guan Xuxu completed the discharge procedures and returned home alone.

[0041] Storyboard 2: As soon as she stepped into the door of Guan’s villa, a suitcase crashed at her feet with a bang.

[0042] Storyboard 3: A beautiful woman dressed elegantly stands at the entrance and looks down at her.

[0043] Storyboard 4: His eyes swept across the girl's delicate features and snow-white skin, and a hint of jealousy flashed in his eyes, followed by undisguised disgust.

[0044] In a possible way, in order to more accurately identify sentence boundaries and make the split storyboard copy retain semantic integrity, after obtaining the target text, the punctuation in the target text can be standardized or corrected, and then the target text is split to obtain multiple storyboard copies. In this way, the storyboard copy can better utilize context information, improve the coherence of the storyboard and the fluency of the story, and thus improve the video quality.

[0045] After obtaining multiple storyboards, you can generate multiple storyboard images for each storyboard.

[0046] For example, for each storyboard copy, the storyboard copy can be input into the first machine learning model used to generate prompt words, so as to obtain the prompt words corresponding to the storyboard copy, and then by adjusting the random seed parameter value seed of the machine learning model used to generate pictures, and inputting the prompt words into the second machine learning model, multiple storyboard pictures corresponding to the prompt words are obtained, that is, multiple storyboard pictures corresponding to the storyboard copy. The value range of seed is The specific value can be determined according to the number of required storyboard pictures, and the embodiments of the present disclosure do not impose any limitation on this.

[0047] It should be understood that since the prompt words are used for the machine learning model to generate storyboard images, the accuracy of the prompt words directly affects the matching degree between the generated storyboard images and the storyboard text. Therefore, in order to ensure the accuracy and completeness of the prompt words, and thereby make the generated storyboard images match the storyboard text, in a possible manner, when the prompt words corresponding to the storyboard text are generated by the first machine learning model, the first k sentences of the storyboard text and the last s sentences of the storyboard text of the target storyboard text can be input into the machine learning model together, so that the first machine learning model can write the prompt in combination with the context, making the prompt more accurate, so that the storyboard images generated based on the prompt can match the storyboard text, reducing the number of times the storyboard images are regenerated, and thereby improving the efficiency of video production. Wherein, k and s are positive integers, k and s can be the same or different, and the embodiments of the present disclosure do not impose any restrictions on this.

[0048] For example, continuing to refer to the above example, when the prompt word of the storyboard copy "Just after stepping into the gate of Guan's villa, a suitcase hit her feet with a "clang" sound" is generated by the first machine learning model, a target prompt word for generating storyboard copy prompt words can be generated based on the storyboard copy "Not long after the June college entrance examination, Guan Xuxu went home alone after completing the discharge procedures", "Just after stepping into the gate of Guan's villa, a suitcase hit her feet with a "clang" sound", "The beautiful woman dressed in noble clothes stood at the entrance and looked down at her, her eyes swept across the girl's delicate features and snow-white skin, a trace of jealousy flashed in her eyes, followed by undisguised disgust". For example, the target prompt word can be generated: Please combine the above " The target prompt words are then input into the first machine learning model to obtain the storyboard copy "Just after entering the gate of Guan's villa, a suitcase hit her feet with a "clang" sound", and the prompt words "A young girl with delicate features and fair skin stood at the entrance of a luxurious villa with a suitcase at her feet", such as Figure 2 shown.

[0049] In a possible way, in order to generate a storyboard picture again using the prompt word when the currently generated storyboard picture does not meet the user's expectations, and to make the generated result different from the historical generated result, the seed of the second machine learning model can be adjusted to be different from the historical seed, and then the prompt word is input into the second machine learning model to obtain a different storyboard picture. That is to say, in a possible way, for each storyboard copy, generating multiple storyboard pictures can include: For each storyboard copy, determine the prompt words used to generate the storyboard pictures, and determine the random seed parameter value of the machine learning model, wherein the machine learning model is used to output multiple storyboard pictures according to the random seed parameter value and the input prompt words, and the random seed parameter value is different from the historical random seed parameter value, and the historical random seed parameter value is the random seed parameter of the machine learning model that last generated the storyboard pictures according to the storyboard copy; input the prompt words into the machine learning model to obtain multiple storyboard pictures.

[0050] For example, continuing to refer to the above example, after obtaining the storyboard copy, the storyboard copy "The June college entrance examination has just ended, Guan Xuxu has completed the discharge procedures and returned home alone" can be automatically input into the first machine learning model to obtain the prompt word corresponding to the storyboard copy: a young woman with a delicate face and fair skin, holding a discharge certificate and walking out of the hospital. After that, the seed of the second machine learning model can be randomly set to n, and the prompt word a young woman with a delicate face and fair skin, holding a discharge certificate and walking out of the hospital can be automatically input into the second machine learning model, thereby outputting a corresponding number of storyboard pictures, which can be displayed in the display device of the terminal device. If the currently generated storyboard picture does not meet the user's expectations, the seed of the second machine learning model can be manually adjusted, for example, to m, and then the prompt word a young woman with a delicate face and fair skin, holding a discharge certificate and walking out of the hospital can be input into the second machine learning model, so that different storyboard pictures can be displayed in the display device of the terminal device, where n and m are both positive integers, and m≠n.

[0051] In this embodiment, the number of storyboard images generated by different storyboard texts may be the same or different, and the disclosed embodiment does not impose any limitation on this.

[0052] For example, when it is necessary to control different storyboards to generate the same number of storyboard images, the random seed parameter value of the machine learning model can be controlled to be the same, so that after the prompt words corresponding to each storyboard are input into the machine learning model, the same number of storyboard images can be obtained. When it is necessary to control different storyboards to generate different numbers of storyboard images, the random seed parameter value of the machine learning model can be controlled to be different, that is, the random seed parameter value can be adjusted according to the number of storyboard images required for different storyboards, so that when each storyboard is input into the machine learning model, a different number of storyboard images can be obtained.

[0053] In a possible way, in order to enable the machine learning model to generate different types of storyboard pictures, when generating storyboard pictures through the machine learning model, in addition to inputting prompt words into the machine learning model, the picture type can also be input into the machine learning model, so that the machine learning model can output different types of storyboard pictures, for example, it can output comic storyboard pictures, or it can output two-dimensional storyboard pictures, and then generate different types of videos, further enriching the video types.

[0054] In a possible way, in order to make the generated storyboard pictures more consistent with the meaning expressed by the storyboard copy, in a possible way, the generated storyboard pictures can also be dynamic pictures. Or a video clip that conforms to the storyboard copy can be directly generated through a machine learning model.

[0055] S103: According to a preset grouping strategy, the storyboard images generated by each storyboard copy are divided into a plurality of picture groups, wherein one picture group includes one storyboard image generated by each storyboard copy, and the storyboard images corresponding to the same storyboard copy in different picture groups are different.

[0056] For example, continue to refer to Figure 2 As shown, if each storyboard text generates 2 storyboard pictures, the 6 storyboard pictures can be randomly divided into two groups, and each group includes a storyboard picture corresponding to the storyboard 1 text, a storyboard picture corresponding to the storyboard 2 text, and a storyboard picture corresponding to the storyboard 3 text.

[0057] S104: Generate a target video based on at least one storyboard picture in at least one picture group among the multiple picture groups.

[0058] For example, continuing to refer to the above example, two videos with the same storyline but different picture contents can be generated based on each group of storyboard pictures.

[0059] Through the above technical scheme, multiple storyboards can be determined according to the target text, and multiple storyboard pictures can be generated for each storyboard, and the storyboard pictures can be divided into multiple picture groups according to a preset grouping strategy, and at least the target video can be generated according to the storyboard pictures in at least one picture group in the multiple picture groups. Thus, the text content of the target text can be quickly converted into video content through an automated process, thereby reducing the difficulty and operation complexity of video production and improving video production efficiency. In addition, since a picture group can include a storyboard picture generated by each storyboard, and the storyboard pictures corresponding to the same storyboard in different picture groups are different. Thus, when generating a video based on the storyboard pictures in the picture group, the coherence of the video content can be guaranteed, and a variety of video content selections can be provided for users, avoiding the influence of the video quality due to the homogenization of the storyboard pictures.

[0060] To facilitate understanding of the video generation method provided by the present disclosure, possible implementation methods of the present disclosure are described below.

[0061] In a possible manner, each storyboard copy has the same number of storyboard images. According to a preset grouping strategy, the storyboard images generated by each storyboard copy are divided into multiple image groups, which may include: The number of storyboard pictures is determined as the number of picture groups; and according to the number of picture groups, the storyboard pictures corresponding to all storyboard texts are grouped to obtain a plurality of picture groups, wherein the storyboard pictures corresponding to the same storyboard text in different picture groups are different.

[0062] For example, there are three storyboards, recorded as storyboard A, storyboard B, and storyboard C, wherein each storyboard generates three storyboard images, recorded as storyboard image A1, storyboard image A2, storyboard image A3, storyboard image B1, storyboard image B2, storyboard image B3, storyboard image C1, storyboard image C2, and storyboard image C3. Thus, the number of image groups can be determined as 3 groups, and the 3 storyboard images corresponding to each storyboard can be randomly divided into different image groups, thereby obtaining 3 image groups, for example, the following 3 image groups can be obtained: Picture group 1: storyboard picture A1, storyboard picture B3, storyboard picture C2.

[0063] Picture group 2: storyboard picture A2, storyboard picture B2, storyboard picture C1.

[0064] Picture group 3: storyboard picture A3, storyboard picture B1, storyboard picture C3.

[0065] It should be understood that the above-mentioned method of dividing the multiple storyboard images generated by each storyboard copy into multiple image groups according to the preset grouping strategy is only for illustration and is not enough to limit the pair scheme.

[0066] In a possible manner, when the number of storyboard images corresponding to each storyboard copy is the same, the storyboard images generated by each storyboard copy can also be divided into multiple image groups in the following manner: For each storyboard text, a plurality of storyboard images generated by the storyboard text are numbered; storyboard images with the same number among different storyboard images are grouped together, thereby obtaining a plurality of image groups.

[0067] For example, continuing to refer to the above example, there are 3 storyboards, recorded as storyboard A, storyboard B, and storyboard C, wherein each storyboard generates 3 storyboard pictures, recorded as storyboard A-1, storyboard A-2, storyboard A-3, storyboard B-1, storyboard B-2, storyboard B-3, storyboard C-1, storyboard C-2, and storyboard C-3. Since the numbers of storyboard A-1, storyboard B-1, and storyboard C-1 are the same, all 1, storyboard A-1, storyboard B-1, and storyboard C-1 can be divided into one picture group. Similarly, storyboard A-2, storyboard B-2, and storyboard C-2 can be divided into one picture group, and storyboard A-3, storyboard B-3, and storyboard C-3 can be divided into one picture group, thereby obtaining 3 picture groups.

[0068] By the above method, each picture group can include a storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different. Therefore, when generating a video based on the storyboard pictures in the picture group, multiple videos with different video pictures can be generated under the same storyline, thereby providing users with a variety of video picture choices, thereby reducing the occurrence of homogenized videos and improving video quality.

[0069] In a possible manner, different storyboards correspond to different numbers of storyboard images. Accordingly, according to a preset grouping strategy, the storyboard images generated by each storyboard are divided into a plurality of image groups, which may include: The number of storyboard pictures corresponding to the target storyboard copy is determined as the number of picture groups, wherein the target storyboard copy is the storyboard copy with the most corresponding storyboard pictures among all storyboard copies; for each storyboard copy, when the number of storyboard pictures corresponding to the storyboard copy is equal to the number of picture groups, the storyboard pictures corresponding to the storyboard copy are divided into different picture groups, or, when the number of storyboard pictures corresponding to the storyboard copy is less than the number of picture groups, after the storyboard pictures corresponding to the storyboard copy are divided into different picture groups, the storyboard pictures corresponding to the storyboard copy are regrouped to make the number of storyboard pictures in each picture group equal.

[0070] For example, there are three storyboards, recorded as storyboard D, storyboard E, and storyboard F, wherein storyboard D and storyboard E both generate four storyboard images, recorded as storyboard D1, storyboard D2, storyboard D3, storyboard D4, storyboard E1, storyboard E2, storyboard E3, and storyboard E4. Storyboard F generates two storyboard images, recorded as storyboard F1 and storyboard F2. Since storyboard D and storyboard E generate the same number of storyboard images, storyboard D or storyboard E can be used as the target storyboard. At the same time, since the number of storyboard images generated by storyboard D or storyboard E is 4, the number of picture groups can be determined as 4 groups.

[0071] Since the number of storyboard images generated by storyboard copy D and storyboard copy E is the same as the number of picture groups, the four storyboard images generated by storyboard copy D can be randomly divided into different picture groups, so that each picture group can include a storyboard image generated by storyboard copy D, and the four storyboard images generated by storyboard copy E can be randomly divided into different picture groups, so that each picture group can include a storyboard image generated by storyboard copy E. Since the number of storyboard images generated by storyboard copy F is less than the number of picture groups, the two storyboard images generated by storyboard copy F can be randomly divided into two different picture groups first, and then the two storyboard images generated by storyboard copy F can be regrouped to randomly divide the two storyboard images generated by storyboard copy F into two other different picture groups, so that each picture group can include a storyboard image generated by storyboard copy E, thereby obtaining four picture groups, for example, the following four picture groups can be obtained: Picture group 1: storyboard picture D1, storyboard picture E1, storyboard picture F2.

[0072] Picture group 2: storyboard picture D2, storyboard picture E3, storyboard picture F1.

[0073] Picture group 3: storyboard picture D3, storyboard picture E4, storyboard picture F1.

[0074] Picture group 4: storyboard picture D4, storyboard picture E2, storyboard picture F2.

[0075] It should be understood that the above-mentioned method of dividing the multiple storyboard images generated by each storyboard copy into multiple image groups according to the preset grouping strategy is only for illustration and is not enough to limit the pair scheme.

[0076] In a possible manner, when different storyboards correspond to different numbers of storyboard images, the storyboard images generated by each storyboard may be divided into multiple image groups in the following manner: For each storyboard text, firstly, the multiple storyboard images generated by the storyboard text are numbered; then, the first type of storyboards whose number of storyboard images corresponding to the storyboard text is equal to the number of picture groups, and the second type of storyboards whose number of storyboard images corresponding to the storyboard text is less than the number of picture groups are counted, and for the storyboard images corresponding to the first type of storyboards, the storyboard images with the same number are grouped into one group, thereby obtaining multiple initial picture groups. For the storyboard images corresponding to the second type of storyboards, they are sequentially divided into different initial picture groups according to the order of numbers, thereby obtaining multiple picture groups.

[0077] For example, continuing to refer to the above example, there are three storyboards, recorded as storyboard D, storyboard E, and storyboard F, wherein storyboard D and storyboard E both generate four storyboard images, recorded as storyboard D-1, storyboard D-2, storyboard D-3, storyboard D-4, storyboard E-1, storyboard E-2, storyboard E-3, and storyboard E-4. Storyboard F generates two storyboard images, recorded as storyboard F-1 and storyboard F-2. Since storyboard D and storyboard E are first-class storyboards, and storyboard D-1 and storyboard E-1 have the same number, both 1, storyboard D-1 and storyboard E-1 can be grouped together. Similarly, storyboard picture D-2 and storyboard picture E-2 can be grouped together, storyboard picture D-3 and storyboard picture E-3 can be grouped together, and storyboard picture D-4 and storyboard picture E-4 can be grouped together, thereby obtaining 4 initial picture groups. Since storyboard copy F is a storyboard copy of the second type, storyboard picture F-1 can be grouped into the first initial picture group, storyboard picture F-2 can be grouped into the second initial picture group, storyboard picture F-1 can be grouped into the third initial picture group, and storyboard picture F-2 can be grouped into the fourth initial picture group, thereby obtaining the following 4 picture groups: Picture group 1: storyboard picture D1, storyboard picture E1, storyboard picture F1.

[0078] Picture group 2: storyboard picture D2, storyboard picture E2, storyboard picture F2.

[0079] Picture group 3: storyboard picture D3, storyboard picture E3, storyboard picture F1.

[0080] Picture group 4: storyboard picture D4, storyboard picture E4, storyboard picture F2.

[0081] Through the above method, each picture group can include a storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different. Therefore, when generating a video based on the storyboard pictures in the picture group, multiple videos with different video pictures can be generated under the same storyline, thereby providing users with a variety of video picture choices, reducing the occurrence of homogenized videos, and thus improving video quality.

[0082] In a possible manner, the video generation method may further include: Determine the original video material, and extract the audio in the original video material to obtain the target audio; Accordingly, determining the target text for video generation may include: Perform audio recognition on the target audio to obtain the target text for video generation; Accordingly, generating a target video at least according to the storyboard pictures in at least one picture group among the multiple picture groups may include: A target video is generated according to the target audio and the storyboard pictures in at least one of the plurality of picture groups.

[0083] It should be understood that, in the related technology, when making a video, it is necessary to first generate an initial video based on the storyboard picture, and then dub the generated initial video through a dubbing tool, or by additionally uploading an audio file and fusing the audio file and the initial video to obtain the final target video. In this embodiment, the target audio can be extracted from the original video material, and after the target audio is extracted, on the one hand, the target audio can be retained, and on the other hand, a storyboard picture that matches the target audio can be generated based on the target audio. Therefore, when generating a video, the video can be directly generated based on the storyboard picture and the target audio to obtain a target video with consistent sound and picture, without the need to perform additional dubbing based on the video screen content, thereby reducing the operation steps of video production and improving the efficiency of video production.

[0084] In a possible manner, the target audio may include first information, the first information being used to indicate positions of different audio segments in the target audio in the target audio, and accordingly, the storyboard picture may include second information, the second information being used to indicate positions of audio segments corresponding to the storyboard picture in the target audio, and generating a target video according to the target audio and the storyboard picture in at least one picture group among the multiple picture groups may include: According to the first information and the second information, an audio segment corresponding to the storyboard picture is determined; and according to the target audio, the audio segment corresponding to the storyboard picture, and the storyboard picture in at least one of the multiple picture groups, a target video is generated.

[0085] In this embodiment, the first information and the second information can be set according to actual conditions, and the embodiment of the present disclosure does not impose any restrictions on this. For example, different audio segments can be pre-timed according to the start playback time of the target audio, so that the first information can include the timestamps corresponding to different audio segments in the target audio. Correspondingly, when the target audio is converted into a target text, the target text can include the third information, and the third information can include the timestamps of the audio segments corresponding to different texts in the target text, and when the target text is split into multiple storyboards, each storyboard can include the fourth information, and the fourth information can include the timestamp of the audio segment corresponding to the storyboard, so that when the storyboard picture is generated according to the storyboard, each storyboard picture can include the second information, and the second information can include the timestamp of the audio segment corresponding to the storyboard picture. Or different audio segments can be numbered in advance according to their positions in the target audio, so that the first information can include the numbers of different audio segments in the target audio, and accordingly, the second information can include the numbers of the audio segments corresponding to the storyboard picture. Of course, the first information and the second information can also be other information, and the embodiment of the present disclosure does not impose any restrictions on this.

[0086] The following takes the example that the first information may include the timestamps corresponding to different audio segments in the target audio, and the second information may include the timestamps corresponding to the audio segments of the storyboard images to illustrate this embodiment: For example, the target audio clips include 3, recorded as audio clip 1, audio clip 2 and audio clip 3, and the timestamp corresponding to audio clip 1 is 0:30, the timestamp corresponding to audio clip 2 is 2:12, and the timestamp corresponding to audio clip 3 is 3:43. A certain picture group includes 3 storyboard pictures, recorded as storyboard picture 1, storyboard picture 2 and storyboard picture 3, and the timestamp corresponding to storyboard picture 1 is 3:43, the timestamp corresponding to storyboard picture 2 is 0:30, and the timestamp corresponding to storyboard picture 3 is 2:12. Therefore, it can be determined that the audio clip corresponding to storyboard picture 1 is audio clip 3, the audio clip corresponding to storyboard picture 2 is audio clip 1, and the audio clip corresponding to storyboard picture 3 is audio clip 1. Then, when generating a video, the video can be generated based on the target audio and the order of storyboard pictures: storyboard picture 3, storyboard picture 2 and storyboard picture 1.

[0087] Through the above method, the audio clip corresponding to the storyboard picture can be determined, so that when a video is generated according to the target audio and the storyboard pictures in the picture group, each audio clip can be matched with each storyboard picture, thereby achieving audio and video synchronization of the video and further improving the video quality.

[0088] In a possible way, in order to further improve the video quality, after obtaining the storyboard pictures or picture groups, you can also add picture effects, picture stickers, transition effects between storyboard pictures, storyboard subtitles, etc. to each storyboard picture, thereby enriching the generated video content and further reducing the generation of homogeneous videos. Alternatively, after obtaining the picture group, you can also manually add real-life video materials, or digital people generated based on machine learning models for interacting with users, and then generate videos through mixed cutting and splicing, etc., so as to enrich the video content and improve the video quality.

[0089] Based on the same concept, the embodiment of the present disclosure also provides a video generating device, such as Figure 3 As shown, the video generating device 300 may include: A determination module 301 is used to determine a target text for video generation; A first processing module 302 is used to determine a plurality of storyboards according to the target text, and generate a plurality of storyboard images for each storyboard; The second processing module 303 is used to divide the storyboard pictures generated by each storyboard copy into multiple picture groups according to a preset grouping strategy, wherein one picture group includes one storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different; The third processing module 304 is used to generate a target video according to at least one storyboard picture in at least one picture group among the multiple picture groups.

[0090] Through the above-mentioned video generation device 300, multiple storyboards can be determined according to the target text, and multiple storyboard pictures can be generated for each storyboard, and the storyboard pictures can be divided into multiple picture groups according to a preset grouping strategy, and at least the target video can be generated according to the storyboard pictures in at least one picture group in the multiple picture groups. Thus, the text content of the target text can be quickly converted into video content through an automated process, thereby reducing the difficulty of video production and the complexity of operation, and improving the efficiency of video production. In addition, since a picture group can include a storyboard picture generated by each storyboard, and the storyboard pictures corresponding to the same storyboard in different picture groups are different. Thus, when generating a video based on the storyboard pictures in the picture group, the coherence of the video content can be guaranteed, and a variety of video content selections can be provided for users, avoiding the influence of the video quality due to the homogenization of the storyboard pictures.

[0091] In a possible manner, each storyboard copy corresponds to the same number of storyboard images. Accordingly, the second processing module 302 may include: A first determining unit, used to determine the number of storyboard pictures as the number of picture groups; The first grouping unit is used to group the storyboard pictures corresponding to all the storyboard texts according to the number of picture groups to obtain a plurality of picture groups, wherein the storyboard pictures corresponding to the same storyboard text in different picture groups are different.

[0092] In a possible manner, different storyboards correspond to different numbers of storyboard images. Accordingly, the second processing module 302 may include: A second determining unit is used to determine the number of storyboard images corresponding to the target storyboard copy as the number of image groups, wherein the target storyboard copy is the storyboard copy with the most corresponding storyboard images among all the storyboard copies; The second grouping unit is used for, for each storyboard copy, when the number of storyboard pictures corresponding to the storyboard copy is equal to the number of picture groups, dividing the storyboard pictures corresponding to the storyboard copy into different picture groups, or, when the number of storyboard pictures corresponding to the storyboard copy is less than the number of picture groups, after dividing the storyboard pictures corresponding to the storyboard copy into different picture groups, re-grouping the storyboard pictures corresponding to the storyboard copy so that the number of storyboard pictures in each picture group is equal.

[0093] In a possible manner, the video generating device 300 may further include: The fourth processing module is used to determine the original video material and extract the audio in the original video material to obtain the target audio; Accordingly, the determination module 301 may be used to perform audio recognition on the target audio to obtain the target text for video generation; Accordingly, the third processing module 303 may be configured to generate a target video according to the target audio and storyboard pictures in at least one of the multiple picture groups.

[0094] In a possible manner, the target audio includes first information, and the first information is used to indicate the positions of different audio segments in the target audio. Accordingly, the storyboard picture includes second information, and the second information is used to indicate the positions of the audio segments corresponding to the storyboard picture in the target audio. Accordingly, the third processing module 303 may include: A third determining unit, configured to determine an audio segment corresponding to the storyboard picture according to the first information and the second information; The first processing unit is used to generate a target video according to the target audio, the audio segment corresponding to the storyboard picture, and the storyboard picture in at least one picture group among the multiple picture groups.

[0095] In a possible manner, the first information includes timestamps corresponding to different audio segments in the target audio, and the second information includes timestamps corresponding to the audio segments of the storyboard images.

[0096] In a possible manner, the first processing module 301 may include: The fourth determination unit is used to determine the prompt words used to generate the storyboard pictures for each storyboard copy, and to determine the random seed parameter value of the machine learning model, wherein the machine learning model is used to output multiple storyboard pictures according to the random seed parameter value and the input prompt words, and the random seed parameter value is different from the historical random seed parameter value, and the historical random seed parameter value is the random seed parameter value of the machine learning model when it last generated the storyboard picture according to the storyboard copy.

[0097] The second processing unit is used to input the prompt words into the machine learning model to obtain multiple storyboard pictures.

[0098] Based on the same concept, an embodiment of the present disclosure further provides a computer-readable medium on which a computer program is stored. When the program is executed by a processing device, the steps of any of the above-mentioned video generation methods are implemented.

[0099] Based on the same concept, the present disclosure also provides an electronic device, which may include: a storage device having a computer program stored thereon; The processing device is used to execute the computer program in the storage device to implement the steps of any of the above-mentioned video generation methods.

[0100] Based on the same concept, an embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned video generation methods when executed by a processor.

[0101] Reference below Figure 4 , which shows a schematic diagram of the structure of an electronic device 400 suitable for implementing the embodiment of the present disclosure. The terminal device in the embodiment of the present disclosure may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0102] like Figure 4As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 402 or a program loaded from a storage device 408 into a random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 are also stored. The processing device 401, the ROM 402, and the RAM 403 are connected to each other via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0103] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0104] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the ROM 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0105] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer readable signal media may also be any computer readable medium other than computer readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0106] In some embodiments, any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol) can be used for communication, and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0107] The computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.

[0108] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: determines a target text for video generation; determines multiple storyboards according to the target text, and generates multiple storyboard pictures for each storyboard; divides the storyboard pictures generated by each storyboard into multiple picture groups according to a preset grouping strategy, wherein one picture group includes one storyboard picture generated by each storyboard, and different storyboard pictures corresponding to the same storyboard in different picture groups are different; generates a target video at least based on the storyboard pictures in at least one of the multiple picture groups.

[0109] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0110] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0111] The modules involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a module does not, in some cases, limit the module itself.

[0112] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0113] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0114] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0115] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0116] Although the subject matter has been described in language specific to structural features and / or method logic actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims. Regarding the device in the above embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Claims

1. A video generation method, characterized in that: The video generation method comprises: Determine the target text for video generation; Determine a plurality of storyboards according to the target text, and generate a plurality of storyboard images for each of the storyboards; According to a preset grouping strategy, the storyboard pictures generated by each storyboard copy are divided into a plurality of picture groups, wherein one picture group includes one storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different; A target video is generated based on at least one storyboard picture in at least one of the plurality of picture groups.

2. The video generation method according to claim 1, characterized in that: The number of storyboard pictures corresponding to each storyboard copy is the same, and the storyboard pictures generated by each storyboard copy are divided into a plurality of picture groups according to a preset grouping strategy, including: Determine the number of storyboard pictures as the number of picture groups; According to the number of picture groups, the storyboard pictures corresponding to all storyboard texts are grouped to obtain a plurality of picture groups, wherein the storyboard pictures corresponding to the same storyboard text in different picture groups are different.

3. The video generation method according to claim 1, characterized in that: Different storyboards correspond to different numbers of storyboard images. According to a preset grouping strategy, the storyboard images generated by each storyboard are divided into a plurality of image groups, including: Determine the number of storyboard images corresponding to the target storyboard copy as the number of image groups, wherein the target storyboard copy is the storyboard copy with the most corresponding storyboard images among all storyboard copies; For each of the storyboards, when the number of storyboard pictures corresponding to the storyboard is equal to the number of picture groups, the storyboard pictures corresponding to the storyboard are divided into different picture groups; or, when the number of storyboard pictures corresponding to the storyboard is less than the number of picture groups, after the storyboard pictures corresponding to the storyboard are divided into different picture groups, the storyboard pictures corresponding to the storyboard are grouped again to make the number of storyboard pictures in each of the picture groups equal.

4. The video generation method according to any one of claims 1 to 3, characterized in that: The video generation method further includes: Determine an original video material, and extract audio from the original video material to obtain target audio; The step of determining a target text for video generation includes: Performing audio recognition on the target audio to obtain a target text for video generation; The step of generating a target video based on at least one storyboard picture in at least one picture group among the plurality of picture groups comprises: A target video is generated according to the target audio and the storyboard pictures in at least one of the plurality of picture groups.

5. The video generation method according to claim 4, characterized in that: The target audio includes first information, the first information is used to indicate the positions of different audio segments in the target audio in the target audio, and correspondingly, the storyboard picture includes second information, the second information is used to indicate the positions of the audio segments corresponding to the storyboard picture in the target audio, and generating a target video according to the target audio and the storyboard picture in at least one of the plurality of picture groups, comprises: Determining an audio clip corresponding to the storyboard picture according to the first information and the second information; A target video is generated according to the target audio, the audio segment corresponding to the storyboard picture, and the storyboard picture in at least one of the plurality of picture groups.

6. The video generation method according to claim 5, characterized in that: The first information includes timestamps corresponding to different audio segments in the target audio, and the second information includes timestamps corresponding to the audio segments of the storyboard images.

7. The video generation method according to any one of claims 1 to 3, characterized in that: For each of the storyboards, a plurality of storyboard images are generated, including: For each of the storyboards, determining a prompt word for generating the storyboard picture, and determining a random seed parameter value of a machine learning model, wherein the machine learning model is used to output a plurality of storyboard pictures according to the random seed parameter value and the input prompt word, and the random seed parameter value is different from a historical random seed parameter value, and the historical random seed parameter value is the random seed parameter value of the machine learning model when it last generated the storyboard picture according to the storyboard; The prompt words are input into the machine learning model to obtain multiple storyboard pictures.

8. A video generating device, characterized in that: The video generating device comprises: A determination module, used for determining a target text for video generation; A first processing module is used to determine a plurality of storyboards according to the target text, and generate a plurality of storyboard images for each of the storyboards; A second processing module is used to divide the storyboard pictures generated by each storyboard copy into a plurality of picture groups according to a preset grouping strategy, wherein one picture group includes one storyboard picture generated by each storyboard copy, and the storyboard pictures corresponding to the same storyboard copy in different picture groups are different; The third processing module is used to generate a target video based on at least the storyboard pictures in at least one picture group among the plurality of picture groups.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processing device, the steps of the method according to any one of claims 1 to 7 are implemented.

10. An electronic device, characterized in that: include: a storage device having a computer program stored thereon; A processing device, configured to execute the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images