Video generation method and device, equipment and medium

Abstract

The embodiment of the invention relates to a video generation method and device, equipment and a medium, and the method comprises the steps: obtaining video basic data which comprises a scene identifier, a target script and a plurality of first videos; based on the target script and scene information corresponding to the scene identifier, generating a split document and a split audio; performing video screening and duration zooming processing on the plurality of first videos according to the sub-mirror dimension based on the sub-mirror audio to obtain a plurality of second videos; generating split subtitles based on the split audio and the split copywriting; and generating a target video based on the plurality of second videos, the plurality of split audios and the plurality of split subtitles. According to the method and the device, a complete and efficient automatic video generation process is realized, the functions are comprehensive, the usability is high, a large amount of labor time and energy are saved, the video generation efficiency is greatly improved, the matching degree of the video and the audio is effectively improved through screening of the split videos and duration zooming processing, and then the video quality of the generated video is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of video processing technology, and in particular to a video generation method, device, equipment and medium. Background Art

[0002] Video is an important means of displaying an item, and video quality plays a key role in improving the display effect. Currently, the production of high-quality videos requires video shooting and editing capabilities, and requires a lot of manual time and effort, which is inefficient. In order to solve the above problems, related technologies can use deep learning technology to optimize some steps of video generation, but video generation still cannot meet the requirements of multiple dimensions such as ease of use, efficiency, and video quality at the same time, and needs to be improved. Summary of the invention

[0003] In order to solve the above technical problems, the present disclosure provides a video generation method, device, equipment and medium.

[0004] The present disclosure provides a video generation method, the method comprising:

[0005] Acquire basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos;

[0006] Generate storyboard text and storyboard audio based on the target script and the scene information corresponding to the scene identifier;

[0007] Based on the storyboard audio, the plurality of first videos are screened and time-scaled according to the storyboard dimension to obtain a plurality of second videos;

[0008] Generate storyboard subtitles based on the storyboard audio and the storyboard copy;

[0009] A target video is generated based on the multiple second videos, the multiple storyboard audios, and the multiple storyboard subtitles.

[0010] The present disclosure also provides a video generation device, the device comprising:

[0011] An acquisition module, used for acquiring basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos;

[0012] A storyboard module, used for generating storyboard text and storyboard audio based on the target script and the scene information corresponding to the scene identifier;

[0013] A video editing module, configured to perform video screening and time-scaling processing on the plurality of first videos according to the storyboard dimensions based on the storyboard audio to obtain a plurality of second videos;

[0014] A subtitle module, used for generating storyboard subtitles based on the storyboard audio and the storyboard copy;

[0015] A video generation module is used to generate a target video based on the multiple second videos, the multiple storyboard audios and the multiple storyboard subtitles.

[0016] An embodiment of the present disclosure also provides an electronic device, comprising: a processor; a memory for storing executable instructions of the processor; the processor is used to read the executable instructions from the memory and execute the instructions to implement the video generation method provided in the embodiment of the present disclosure.

[0017] The embodiment of the present disclosure further provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the video generation method provided by the embodiment of the present disclosure.

[0018] The technical solution provided by the embodiments of the present disclosure has the following advantages over the prior art: the video generation solution provided by the embodiments of the present disclosure obtains basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos; based on the target script and the scene information corresponding to the scene identifier, a storyboard text and a storyboard audio are generated; based on the storyboard audio, the plurality of first videos are screened and time-scaled according to the storyboard dimension to obtain a plurality of second videos; storyboard subtitles are generated based on the storyboard audio and the storyboard text; a target video is generated based on the plurality of second videos, the plurality of storyboard audios, and the plurality of storyboard subtitles. By adopting the above technical solution, storyboard text and storyboard audio can be automatically generated based on the scene information corresponding to the scene identifier in the video basic data and the target script, and then storyboard subtitles can be generated. The video can be screened and time-scaled through the storyboard audio. Finally, the final target video is generated based on the processed video, storyboard audio and storyboard subtitles, realizing a complete and efficient automated video generation process with comprehensive functions and high ease of use, saving a lot of manual time and effort, greatly improving the video generation efficiency, and the text and audio generated based on the scene information are more in line with the actual application scenarios. The screening and time-scaling of the storyboard video can effectively improve the matching degree of the video and audio, thereby improving the video quality of the generated video. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

[0020] Figure 1 A schematic diagram of a flow chart of a video generation method provided by an embodiment of the present disclosure;

[0021] Figure 2 A schematic diagram of a copywriting generation process provided by an embodiment of the present disclosure;

[0022] Figure 3 A schematic diagram of an audio generation process provided by an embodiment of the present disclosure;

[0023] Figure 4 A flowchart of another video generation method provided by an embodiment of the present disclosure;

[0024] Figure 5 A schematic diagram of a video generation process provided by an embodiment of the present disclosure;

[0025] Figure 6 A schematic diagram of the structure of a video generating device provided by an embodiment of the present disclosure;

[0026] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0028] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0029] The term "including" and its variations used herein are open inclusions, i.e., "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". The relevant definitions of other terms will be given in the following description.

[0030] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0031] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0032] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0033] In order to improve the quality of videos, deep learning tools can be used to provide support and promote the rapid generation of videos. However, there are the following defects: without professional video shooting and editing capabilities, high-quality video content cannot be produced; the process of video production may be cumbersome and require a lot of time and effort, resulting in low efficiency in video content production; the one-click video algorithm in related technologies cannot adapt to various application scenarios, such as vehicle scenarios; the one-click video algorithm in related technologies has a relatively rough presentation effect, which is specifically manifested in poor matching between pictures and audio, stiff copywriting, unnatural dubbing, etc. Video generation still cannot meet the requirements of multiple dimensions such as ease of use, efficiency, and video quality at the same time, and needs to be improved.

[0034] In order to solve the above problems, the embodiments of the present disclosure provide a video generation method, which is introduced below in conjunction with specific embodiments.

[0035] Figure 1 The flowchart of a video generation method provided by an embodiment of the present disclosure is shown in FIG. 1 , which can be executed by a video generation device, wherein the device can be implemented by software and / or hardware and can generally be integrated in an electronic device. Figure 1 As shown, the method includes:

[0036] Step 101: Obtain basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos.

[0037] Among them, the video basic data can be the data input by the user when he needs to generate a video or make a video. The specific data included in the video basic data varies according to the usage scenario. The embodiment of the present disclosure takes the vehicle scene as an example. The video basic data can include scene identification, target script and first video. The video basic data can also include dubbing parameters, transition parameters and background music, etc., which are set according to actual conditions.

[0038] The scene identifier may be information that indicates which specific usage scene the video is currently being generated for. For example, for a vehicle scene, the scene identifier may be a vehicle identifier. The vehicle identifier may be information that indicates which vehicle a video is currently being generated for. The vehicle identifier may include the vehicle model, the name or trademark of the vehicle manufacturer, etc. The target script may be a text that provides detailed guidance and structural requirements for the content and style of the target video to be generated. The video presentation method may be planned in detail, including but not limited to video style, timeline, dialogue, narration, scene setting, action guidance, camera switching, visual effects, background music and sound effects, etc. The target script may include multiple storyboards. A storyboard is the smallest unit for shooting characters, actions or events in a video. It provides detailed text descriptions and picture composition for the shooting content, angle, movement method, etc. of a shot.

[0039] The first video may be an original video that needs to be edited or processed, and may be uploaded by a user. There may be multiple first videos.

[0040] Specifically, when a user needs to make a video, he can first input basic video data, and the video generating device can obtain the basic video data. Exemplarily, after obtaining the scene identifier selected by the user, the video generating device can first display multiple preset scripts, determine the target script from the multiple preset scripts in response to the user's selection operation, and then display the target script including multiple video upload areas corresponding to the storyboard scripts. One storyboard script can correspond to one video upload area, and each video upload area displays the video requirement text that the storyboard script needs to be uploaded by the user. The user can upload the first video corresponding to the accurate storyboard script according to the video requirement text, and obtain multiple first videos corresponding to one storyboard script, and finally obtain multiple first videos corresponding to the target script. By constraining the user to upload the original video of the corresponding type of the storyboard script by the storyboard dimension, for example, in a vehicle scene, the storyboard script explains the interior of the vehicle and requires the user to upload a video related to the vehicle interior, so that the subsequently generated video screen content matches the text corresponding to the script.

[0041] Optionally, after acquiring the multiple first videos, the video generation device may also preprocess the multiple first videos. The preprocessing may include at least one of video anti-shake processing, color space processing, audio processing, etc. The video anti-shake processing may be implemented using a video anti-shake algorithm for the shaking video in the first video; the color space processing may be performed on the video in the High Dynamic Range Imaging (HDR) format in the first video, and converted to the Standard Dynamic Range (SDR) color gamut; the audio processing may be to split the video from the beginning of the audio stream for the video that selects to retain the original sound, cut off the front audioless segment, and remove the useless "air port" part. By preprocessing the video, the video quality can be improved, which helps to improve the quality of subsequent video generation.

[0042] Step 102: Generate storyboard text and storyboard audio based on the target script and scene information corresponding to the scene identifier.

[0043] The scene information may be the introduction information, attribute information and other information related to the specific scene corresponding to the scene identifier. For example, for a vehicle scene, the scene information may be vehicle information, and the vehicle information may be all the information related to a vehicle. For example, the vehicle information may include but is not limited to the basic information such as the vehicle model, color, usage nature, displacement, power, chassis number, inspection information, frame number, vehicle model and style, etc. The storyboard copy may be the copy content after the scene information is filled in for a storyboard script. The storyboard copy may describe the content, action, photography director, role performance, dialogue and sound effects of a shot. The storyboard audio may be the dubbing audio generated according to the storyboard copy. The embodiment of the present disclosure takes the automatic generation of the storyboard audio as an example. In the actual scene, the first video may include the audio of the user's explanation as the storyboard audio. The storyboard script, the storyboard copy and the storyboard audio are divided according to the storyboard dimension and correspond one to one, that is, one storyboard script corresponds to one storyboard copy and one storyboard audio.

[0044] After the video generation device obtains the target script and scene identifier, it can obtain scene information through the interface according to the scene identifier, and obtain the target copy template corresponding to the target script, fill the scene information into the target copy template to obtain multiple storyboards; or it can use the model to generate multiple storyboards. Then, based on the multiple storyboards, the model is used to generate the corresponding multiple storyboard audios.

[0045] In some embodiments, generating storyboard text and storyboard audio based on a target script and scene information corresponding to a scene identifier may include: obtaining a prompt word template corresponding to the target script; filling the scene information corresponding to the scene identifier into the prompt word template to obtain text prompt words; inputting the text prompt words into a text generation model to output a target text, wherein the target text includes multiple storyboard texts; and generating target audio with a preset tone using a tone reproduction algorithm based on the target text, wherein the target audio includes multiple storyboard audios.

[0046] The prompt word template may be a template for configuring the content and position included in the prompt word, and is used to generate a prompt word (Prompt). The prompt word may be a text or sentence fragment used to trigger and guide the model to perform a specific task to generate a specific output content. The prompt word may express the task that the model should perform to guide the model to generate specific text, image, audio and other content. The prompt word template of the disclosed embodiment is used to generate a copy prompt word. The prompt word template includes a plurality of placeholders for filling in different information, such as filling in scene information. The copy prompt word may be a prompt word for guiding the copy generation model to generate a target copy. The copy generation model may be a model for generating a video copy, and the specific type is not limited. For example, the copy generation model may be a deep learning model, a pre-trained model based on large-scale data, and the like. The target copy may be a copy corresponding to a target script, and may include multiple storyboard copies. The target audio may be the audio corresponding to the target video to be generated, and may include multiple storyboard audios.

[0047] Specifically, when generating storyboard text and storyboard video, the video generating device can first obtain a prompt word template corresponding to the target script, where the prompt word template is preset, and different scripts are provided with corresponding prompt word templates. Then, the scene information can be filled into the corresponding position in the prompt word template to obtain the text prompt word; the text prompt word is input into the text generation model for inference, and a target text including multiple storyboard texts is output; then, a timbre replication algorithm is used to replicate the preset timbre of the speaker based on the preset audio uploaded by the user, or to determine the preset timbre based on the user's selection from multiple local timbres, and a target audio consistent with the preset timbre is generated according to the target text. The preset audio can be an audio uploaded by the user including the user's own voice or the voice of other people, and the timbre replication algorithm can be a technology for replicating the timbre, speaking speed, short sentence characteristics, etc. of a specific speaker's voice.

[0048] Exemplarily, the copy prompt word may include format control items of the target copy, as shown below: "Please output the results according to the above rules and the following format. The return format is as follows, "{xxx}" represents the placeholder: Storyboard 1\{copy}; Storyboard 2\{copy}..." This is just an example.

[0049] For example, Figure 2A schematic diagram of a copywriting generation process provided by an embodiment of the present disclosure, such as Figure 2 As shown in the figure, the copy generation process may include: judging whether the target script includes the corresponding target copy template, if so, grabbing the scene information and filling it into the target copy template to produce a target copy including multiple storyboard copies; otherwise, grabbing the scene information and filling the prompt word template, calling the copy production model to generate a target copy including multiple storyboard copies.

[0050] For example, Figure 3 A schematic diagram of an audio generation process provided by an embodiment of the present disclosure is shown in FIG. Figure 3 As shown in the figure, the audio production process may include: determining whether to replicate the timbre, which can be determined based on the user's choice. If so, providing a preset audio to replicate the preset timbre; otherwise, providing multiple local timbres to determine the preset timbre based on the user's choice; and then producing the target audio according to the preset timbre based on the target text.

[0051] In the above solution, text and audio are automatically generated based on scene information, which is more in line with actual application scenarios and effectively solves the problem of stiff text. Real human voices are cloned as dubbing for videos to enhance realism.

[0052] Step 103: Perform video screening and time-scaling processing on the multiple first videos according to the storyboard dimension based on the storyboard audio to obtain multiple second videos.

[0053] The second video may be a video obtained after filtering and scaling the first video, and there may be multiple second videos.

[0054] The video generating device first divides the multiple first videos according to the storyboard dimension, determines the multiple first videos corresponding to each storyboard audio, and then selects a certain number of first videos with longer duration from the multiple first videos included in the storyboard according to the audio duration of the storyboard audio and performs duration scaling processing, so that the total duration of the selected first videos is equal to the audio duration of the storyboard audio. The number of first videos selected during the above video screening can be determined according to actual conditions.

[0055] For example, Figure 4 A flowchart of another video generation method provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, in a feasible implementation manner, for each storyboard audio, the above step 103 may include the following steps:

[0056] Step 401: Determine the number of videos to be extracted and the target duration based on the audio duration and duration threshold of the storyboard audio.

[0057] Among them, the audio duration can be the specific duration corresponding to the storyboard audio, and the duration threshold can be the minimum duration of the second video determined according to business needs, that is, the duration of the second video needs to be greater than or equal to the duration threshold to avoid the video duration being too short and unusable. For example, the duration threshold can be set to 2 seconds. The number of extracted videos can be the specific number of first videos screened for a storyboard determined for multiple first videos included in a storyboard, and the number of prompt videos corresponding to different storyboards can be different. The target duration can be the corresponding duration to which each first video required to be scaled according to the audio duration of the storyboard audio is determined for multiple first videos included in a storyboard. The target duration can include multiple scaled durations, and the scaled duration can be the specific duration after the duration of a first video is scaled, and the scaled duration can be determined according to the audio duration of the storyboard audio.

[0058] The video generating device traverses in descending order from the number of videos of the multiple first videos corresponding to the storyboard audio as the number of extracted videos according to the audio duration of the storyboard audio. When a certain value is found as the number of extracted videos, if each scaled duration in the target duration corresponding to the multiple first videos is greater than or equal to the duration threshold, then the value is the number of extracted videos and the target duration is also determined.

[0059] In some embodiments, determining the number of videos to be extracted and the target duration based on the audio duration and duration threshold of the storyboard audio may include: determining the number of videos of multiple first videos corresponding to the storyboard audio; determining the number of videos as the number to be processed, and judging whether the scaled duration of each first video extracted according to the number to be processed and the audio duration is greater than or equal to the duration threshold; if so, determining the number to be processed as the number of videos to be extracted, and determining the combination of the scaled durations of each extracted first video as the target duration; otherwise, determining the value of the number of videos minus one as the new number to be processed and returning to continue judging until the number to be processed is one.

[0060] Among them, the number to be processed can be a calculated value used as a possible number of extracted videos to determine whether it can meet the duration threshold. The number to be processed decreases in reverse order from the number of videos of multiple first videos corresponding to the current storyboard audio until a certain value is determined to be the number of extracted videos or the number to be processed is 1.

[0061] The video generation device can first divide the multiple first videos according to the storyboard dimension. Assuming that there are f storyboards in total, the audio duration of the i-th storyboard audio can be expressed as Assuming that the i-th storyboard includes m first videos, that is, the i-th storyboard audio corresponds to m first videos, the first video can be expressed as The corresponding original duration is expressed as The number of videos is m, assuming the number to be processed is m s , 1<<ms <<m, then after video screening and video scaling according to the number of videos to be processed and the audio duration, the first video Zoom duration It can be expressed as: The number to be processed is traversed in descending order starting from m, and it is determined whether the scaled duration of all the first videos corresponding to the number to be processed is greater than or equal to the duration threshold. If a number to be processed is found to meet the requirement, the number to be processed at this time is determined as the number of extracted videos, and the scaled duration of each first video corresponding to the number to be processed is determined as the target duration; if the number to be processed is traversed to 1, 1 is directly used as the number of extracted videos, and the target duration only includes one scaled duration as the audio duration.

[0062] Step 402: extract corresponding multiple third videos from multiple first videos corresponding to the storyboard audio according to the number of extracted videos.

[0063] Among them, the third video can be a first video extracted from multiple first videos included in the current storyboard according to the number of extracted videos, and the number of third videos is the number of extracted videos.

[0064] The video generation device can extract the first videos of the number of extracted videos that are sorted first from the multiple first videos corresponding to the storyboard audio in a preset order to obtain multiple third videos. The preset order can be a specific extraction order when extracting the multiple first videos, that is, extracting the first videos in a preset order. The preset order includes an upload order or a time sorting result. The upload order can be a sorting order from the front to the back of the specific time when the user uploads each first video. The time sorting result can be a sorting result from the time of each first video in a descending order.

[0065] Step 403: perform time scaling processing on the multiple third videos according to the target time length to obtain multiple second videos corresponding to the storyboard audio.

[0066] The duration scaling process may be a process of shortening or lengthening the duration of a video, which may be achieved by adjusting the playback speed or cutting off the video.

[0067] After the video generating device extracts multiple third videos, since the target duration has been determined in the above embodiment, that is, the scaled duration of each first video has been determined, the scaled duration of each third video is known at this time; the original duration of each third video is adjusted to the corresponding scaled duration through scaling processing, and the corresponding second video can be obtained. After the scaling processing of the multiple third videos corresponding to the storyboard audio, the sum of the durations of the multiple second videos obtained is equal to the audio duration.

[0068] In some embodiments, multiple third videos are time-scaled according to a target duration to obtain multiple second videos corresponding to the storyboard audio, including: for each third video, a comparison duration is determined according to the scaled duration corresponding to the target duration of the third video and preset parameters, and a judgment is made as to whether the original duration of the third video is less than or equal to the comparison duration; if so, the original duration of the third video is adjusted to the scaled duration by adjusting the playback speed to obtain the corresponding second video; otherwise, a video segment of the scaled duration is captured from the third video and determined as the corresponding second video.

[0069] The preset parameter may be a parameter that is multiplied by the zoom duration to determine the comparison duration, and the preset parameter is used to control the subsequent adjustment of the playback speed so that it will not be too slow. The preset parameter may be set according to the actual situation, for example, the preset parameter may be 1.2. The comparison duration may be compared with the original duration of the video to determine the duration of the zoom example method.

[0070] When the video generating device performs scaling processing on multiple third videos, for each third duration, the scaled duration corresponding to the third video is multiplied by a preset parameter to obtain a comparison duration, and it is determined whether the original duration of the third video is less than or equal to the comparison duration. If so, the original duration of the third video can be adjusted to the scaled duration by adjusting the playback speed to obtain the second video. Here, if the original duration of the third video is greater than the scaled duration, the playback speed is adjusted from 1 to a value greater than 1 to shorten the duration. If the original duration of the third video is less than the scaled duration, the playback speed is adjusted from 1 to a value less than 1 to stretch the duration. If the original duration of the third video is greater than the comparison duration, a video segment of the scaled duration can be cut from the 1st second onwards from the third video to determine as the corresponding second video.

[0071] In the above scheme, by first judging how many videos are selected for each storyboard to meet the duration threshold based on the storyboard dimension, the subsequent video generation is prevented from being affected by the short videos. After determining the number of screened videos, the duration after scaling can be determined, and then scaling can be performed quickly in a variety of ways according to the scaled duration, effectively improving the scaling effect, and finally achieving duration matching between the audio and the corresponding video in the storyboard dimension, which helps to improve the duration matching effect of subsequent video generation.

[0072] Step 104: Generate storyboard subtitles based on the storyboard audio and storyboard text.

[0073] Storyboard subtitles can add subtitles and corresponding timestamps to a storyboard according to the storyboard copy.

[0074] Specifically, after determining the storyboard audio and storyboard text, the video generating device can divide the storyboard text into individual sentences, and then add the timestamp of each individual sentence according to the uniform speed strategy based on the audio duration of the storyboard audio, thereby obtaining the storyboard subtitles.

[0075] In some embodiments, generating storyboard subtitles based on storyboard audio and storyboard text may include: dividing the storyboard text into multiple single sentences according to punctuation marks and character quantity thresholds; determining the timestamp corresponding to each single sentence according to a uniform speed strategy based on the audio duration of the storyboard audio; and determining a combination of multiple single sentences and their corresponding multiple timestamps as corresponding storyboard subtitles.

[0076] The character number threshold may be a maximum number of characters specified for each single sentence, which may be set according to actual conditions, for example, the character number threshold may be 15. The uniform speed strategy may be that when adding timestamps, the time of each single sentence is allocated according to the number of characters, and the time of each subtitle is fixed. The more characters there are, the longer the time of the single sentence is. The timestamp corresponding to a single sentence may be the time period corresponding to the audio duration of the single sentence in the storyboard audio, which may include the start time and the end time. The time periods corresponding to the timestamps of multiple single sentences are the audio duration.

[0077] When the video generating device generates storyboard subtitles, the storyboard text can be first divided into multiple single sentences according to punctuation marks, and then the single sentences with a character number greater than a character number threshold are extracted and further divided from the middle. For all the single sentences after the division, the single sentences with English letters or numbers at the segmentation point are extracted and moved backward to the segmentation point for re-division to obtain the final multiple single sentences. The accuracy of the single sentence division is improved by limiting the number of characters and moving the segmentation point. Then, it is assumed that n single sentences are included, and the number of characters in the single sentence is expressed as s i , it can be seen s f Indicates the total number of characters in the storyboard copy. The audio duration of the storyboard audio is expressed as The start time in the timestamp of the i-th sentence can be expressed as The end time can be expressed as Afterwards, the multiple single sentences and the timestamp corresponding to each single sentence can be determined as the storyboard subtitles of the current storyboard, and the above processing is performed on each storyboard to obtain multiple storyboard subtitles.

[0078] The execution order of step 103 and step 104 is only an example, and they may be executed in parallel, or step 104 may be executed first and then step 103.

[0079] Step 105: Generate a target video based on the multiple second videos, the multiple storyboard audios, and the multiple storyboard subtitles.

[0080] Specifically, after the video generating device obtains multiple storyboard audios, multiple storyboard subtitles, and multiple second videos according to the storyboard dimension, it can perform video synthesis through a video synthesis algorithm. During the synthesis process, other parameters in the video basic data can also be edited to finally obtain the target video. The target video can be a video generated based on the video basic data input by the user.

[0081] In some embodiments, generating a target video based on multiple second videos, multiple storyboard audios, and multiple storyboard subtitles may include: writing the multiple second videos, multiple storyboard audios, and multiple storyboard subtitles into track editing parameters according to storyboard dimensions, and obtaining the target video using a track editing method according to the track editing parameters.

[0082] Track editing parameters can be various parameters used to adjust the materials on the track when the video is track edited. Track editing parameters can include the position and size, rotation, transparency, color correction, special effects, etc. of the video, audio, and subtitles. By adjusting these parameters, various materials generated by the video can be edited, synthesized, processed with special effects, and other operations to achieve the desired visual effect.

[0083] When generating a target video, the video generating device can construct track editing parameters corresponding to each storyboard according to the requirements of the track editing parameters for multiple second videos, multiple storyboard audios, and multiple storyboard subtitles. In addition, other parameters such as dubbing parameters, transition parameters, and background music in the basic video data can also be written into the track editing parameters in a specified format; the track editing parameters are used to obtain the final target video through callback using the track editing method, and the video is produced by track editing, which effectively improves the video production efficiency.

[0084] The video generation solution provided by the embodiment of the present disclosure obtains basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos; generates a storyboard text and a storyboard audio based on the target script and the scene identifier; performs video screening and time-scaling processing on the plurality of first videos according to the storyboard dimension based on the storyboard audio to obtain a plurality of second videos; generates storyboard subtitles based on the storyboard audio and the storyboard text; and generates a target video based on the plurality of second videos, the plurality of storyboard audios, and the plurality of storyboard subtitles. By adopting the above technical solution, storyboard text and storyboard audio can be automatically generated based on the scene information corresponding to the scene identifier in the video basic data and the target script, and then storyboard subtitles can be generated. The video can be screened and time-scaled through the storyboard audio. Finally, the final target video is generated based on the processed video, storyboard audio and storyboard subtitles, realizing a complete and efficient automated video generation process with comprehensive functions and high ease of use, saving a lot of manual time and effort, greatly improving the video generation efficiency, and the text and audio generated based on the scene information are more in line with the actual application scenarios. The screening and time-scaling of the storyboard video can effectively improve the matching degree of the video and audio, thereby improving the video quality of the generated video.

[0085] In some embodiments, after multiple first videos are screened and time-scaled according to the storyboard dimension based on the storyboard audio to obtain multiple second videos, the video generation may also include: determining the canvas size of the second video with the highest resolution and the largest number of aspect ratios among the multiple second videos as the target canvas size; and scaling the video sizes of the multiple second videos in multiple proportions according to the target canvas size.

[0086] The target canvas size may be a reference size for scaling the video size, that is, the video size needs to be adaptively scaled to the target canvas size for multiple second videos, and the target canvas size may include a canvas width and a canvas height. The video size of the second video may include a video height and a video width. Multi-ratio scaling may be understood as scaling the sizes of multiple second videos at different ratios so as not to compress or stretch a video and cause video deformation.

[0087] After determining multiple second videos in the above embodiment, the video generating device can extract the second video with the highest resolution and the largest number of aspect ratios among the multiple second videos, determine the canvas size of the second video as the target canvas size, scale the video sizes of the multiple second videos to the target canvas size in multiple proportions, and fill the excess parts with black borders.

[0088] Optionally, performing multi-ratio scaling on the video sizes of multiple second videos according to the target canvas size may include: determining the video ratio of each second video and the canvas ratio of the target canvas size; for each second video, determining whether the video ratio of the second video is greater than the canvas ratio, if so, adjusting the video width of the second video to the canvas width of the target canvas size, and adjusting the video height according to the ratio of the canvas width to the video width; otherwise, adjusting the video height of the second video to the canvas height of the target canvas size, and adjusting the video width according to the ratio of the canvas height to the video height.

[0089] The canvas ratio of the target canvas size may be a ratio of the canvas width to the canvas height in the target canvas size, and the video ratio of the second video may be a ratio of the video width to the video height of the second video.

[0090] When the video generating device performs multi-ratio scaling on the sizes of the plurality of second videos, it is assumed that the target canvas size is expressed as (h c’ w c ), the canvas ratio is represented by R c =w c / h c , the video size of the i-th second video can be expressed as (h i’ w i ), i = 1, ..., k, k represents the number of the plurality of second videos, and the video ratio of the i-th second video can be expressed as R i =w i / h i , scale the video size of the i-th second video to the target size, the target size includes the target width and target height, and the target size is expressed as The target size is determined based on the comparison between the video ratio and the brush ratio. The formula is:

[0091]

[0092] When the video ratio of the second video is greater than the canvas ratio, the target width is the canvas width when scaling the second video, and the width ratio of the canvas width to the video width is calculated, and the value obtained by multiplying the width ratio by the video height is determined as the target height; when the video ratio of the second video is less than or equal to the canvas ratio, the target height of the second video is determined as the canvas height, and the height ratio of the canvas height to the video height is calculated, and the value obtained by multiplying the height ratio by the video width is determined as the target width, and then the video width of the second video is adjusted to the target width, and the video height is adjusted to the target height.

[0093] In the above scheme, after processing the video according to the storyboard dimension, each video can be scaled in multiple proportions according to the canvas size of the video with the highest resolution and the largest aspect ratio, so that the video size of each video can be adaptively scaled to the canvas size. Not only does it unify the size of each video, but also when the horizontal and vertical screens are mixed, a video will not be compressed or stretched to cause video deformation.

[0094] Next, the video generation process of the embodiment of the present disclosure is further described by a specific example. Figure 5 A schematic diagram of a video generation process provided by an embodiment of the present disclosure is shown in FIG. Figure 5 As shown, the video generation process in the figure may include: Start. Click on video creation. Enter the video list, script number and other parameters, that is, enter the basic video data. Video preprocessing. Copy generation, that is, generate multiple storyboard copies. Determine whether dubbing is required. If so, perform audio generation, that is, generate multiple storyboard audios, and then perform smart editing; otherwise, perform video text recognition, add subtitles based on the recognized text, and then perform smart editing. Smart editing, that is, perform the video screening, duration scaling processing, storyboard subtitle generation, and video size adjustment of the above-mentioned embodiments. Synthesize the video, that is, finally obtain the above-mentioned target video.

[0095] This solution provides a complete, efficient and intelligent video generation process, integrating intelligent editing, high-quality copy generation, real-person voice simulation, transitions, subtitles, special effects and other functions. It provides users with a simple interface, comprehensive functions and simple operation. People with no editing foundation can easily get started and produce high-quality videos. By restricting users to upload different types of videos based on the storyboard dimension, the picture content and the copy are matched. When generating the copy, high-quality prompt words are designed to solve the problem of stiff copy of large models. The copy transitions naturally, and a variety of copy templates are provided for users to choose. The generated copy can include descriptions of multiple dimensions such as personality, tone, style, and output format. Using the voice replication solution, real human voices are cloned as dubbing for the explanation video. The video effect is natural, and the dubbing is close to real people's voices, which improves the sense of reality.

[0096] Figure 6 FIG. 1 is a schematic diagram of a video generation device provided by an embodiment of the present disclosure. The device can be implemented by software and / or hardware and can generally be integrated into an electronic device. Figure 6 As shown, the device comprises:

[0097] An acquisition module 601 is used to acquire basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos;

[0098] A storyboard module 602, for generating a storyboard text and storyboard audio based on the target script and the scene information corresponding to the scene identifier;

[0099] The video editing module 603 is used to perform video screening and time-scaling processing on the multiple first videos according to the storyboard dimensions based on the storyboard audio to obtain multiple second videos;

[0100] A subtitle module 604, configured to generate storyboard subtitles based on the storyboard audio and the storyboard text;

[0101] The video generation module 605 is used to generate a target video based on the multiple second videos, the multiple storyboard audios and the multiple storyboard subtitles.

[0102] Optionally, the video clipping module 603 includes:

[0103] The first unit is used to determine the number of videos to be extracted and the target duration based on the audio duration and duration threshold of the storyboard audio;

[0104] The second unit is used to extract corresponding multiple third videos from the multiple first videos corresponding to the storyboard audio according to the number of extracted videos;

[0105] The third unit is used to perform time length scaling processing on the multiple third videos according to the target time length to obtain multiple second videos corresponding to the storyboard audio.

[0106] Optionally, the first unit is used to:

[0107] Determine the number of the plurality of first videos corresponding to the storyboard audio;

[0108] Determine the number of videos as the number to be processed, determine whether the scaled duration of each first video extracted according to the number to be processed and the audio duration is greater than or equal to the duration threshold, and if so, determine the number to be processed as the number of extracted videos, and determine the scaled duration combination of each extracted first video as the target duration;

[0109] Otherwise, the value of the video quantity minus one is determined as the new number to be processed and the process returns to continue judging until the number to be processed reaches one.

[0110] Optionally, the third unit is used for:

[0111] For each of the third videos, determine the comparison duration according to the zoomed duration of the third video corresponding to the target duration and the preset parameters, and determine whether the original duration of the third video is less than or equal to the comparison duration. If so, adjust the original duration of the third video to the zoomed duration by adjusting the playback speed to obtain the corresponding second video;

[0112] Otherwise, a video segment of the zoomed duration is captured from the third video and determined as the corresponding second video.

[0113] Optionally, the subtitle module 604 is used to:

[0114] Dividing the storyboard into multiple single sentences according to punctuation marks and character number thresholds;

[0115] Determine the timestamp corresponding to each of the single sentences according to a uniform speed strategy based on the audio duration of the storyboard audio;

[0116] The multiple single sentences and their corresponding multiple timestamp combinations are determined as corresponding storyboard subtitles.

[0117] Optionally, the device further includes a ratio adjustment module, which is used to: perform video screening and time-length scaling processing on the plurality of first videos according to the storyboard dimensions based on the storyboard audio to obtain a plurality of second videos,

[0118] Determine a canvas size of a second video having the highest resolution and the largest number of aspect ratios among the plurality of second videos as a target canvas size;

[0119] The video sizes of the plurality of second videos are scaled in multiple proportions according to the target canvas size.

[0120] Optionally, the ratio adjustment module is specifically configured to include:

[0121] Determining a video ratio of each of the second videos and a canvas ratio of the target canvas size;

[0122] For each of the second videos, determine whether the video ratio of the second video is greater than the canvas ratio; if so, adjust the video width of the second video to the canvas width of the target canvas size, and adjust the video height according to the ratio of the canvas width to the video width;

[0123] Otherwise, the video height of the second video is adjusted to the canvas height of the target canvas size, and the video width is adjusted according to the ratio of the canvas height to the video height.

[0124] Optionally, the video generation module 605 is used to:

[0125] The multiple second videos, the multiple storyboard audios and the multiple storyboard subtitles are written into the track editing parameters according to the storyboard dimensions, and the target video is obtained by using the track editing method according to the track editing parameters.

[0126] The video generating device provided in the embodiments of the present disclosure can execute the video generating method provided in any embodiment of the present disclosure, and has the corresponding functional modules and beneficial effects of the execution method.

[0127] The embodiments of the present disclosure also provide a computer program product, including a computer program / instruction, which implements the video generation method provided by any embodiment of the present disclosure when the computer program / instruction is executed by a processor.

[0128] Figure 7 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present disclosure. Figure 7 , which shows a schematic diagram of the structure of an electronic device 700 suitable for implementing the embodiment of the present disclosure. The electronic device 700 in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0129] like Figure 7 As shown, the electronic device 700 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage device 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0130] Typically, the following devices may be connected to the I / O interface 705: an input device 706 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 707 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 708 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 709. The communication device 709 may allow the electronic device 700 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 7 The electronic device 700 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0131] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 709, or installed from a storage device 708, or installed from a ROM 702. When the computer program is executed by the processing device 701, the above-mentioned functions defined in the video generation method of the embodiment of the present disclosure are executed.

[0132] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0133] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0134] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0135] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: obtains basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos; generates storyboard text and storyboard audio based on the target script and the scene information corresponding to the scene identifier; performs video screening and time-scaling processing on the plurality of first videos according to the storyboard dimension based on the storyboard audio to obtain a plurality of second videos; generates storyboard subtitles based on the storyboard audio and the storyboard text; and generates a target video based on the plurality of second videos, the plurality of storyboard audios, and the plurality of storyboard subtitles.

[0136] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0137] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0138] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware, wherein the name of a unit does not, in some cases, limit the unit itself.

[0139] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0140] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0141] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0142] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0143] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0144] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A video generation method, characterized in that: include: Acquire basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos; Generate storyboard text and storyboard audio based on the target script and the scene information corresponding to the scene identifier; Based on the storyboard audio, the plurality of first videos are screened and time-scaled according to the storyboard dimension to obtain a plurality of second videos; Generate storyboard subtitles based on the storyboard audio and the storyboard copy; A target video is generated based on the multiple second videos, the multiple storyboard audios, and the multiple storyboard subtitles.

2. The method according to claim 1, characterized in that Based on the storyboard audio, the plurality of first videos are screened and time-scaled according to the storyboard dimension to obtain a plurality of second videos, including: Determine the number of videos to be extracted and the target duration based on the audio duration and duration threshold of the storyboard audio; Extracting corresponding multiple third videos from the multiple first videos corresponding to the storyboard audio according to the number of extracted videos; The multiple third videos are time-scaled according to the target duration to obtain multiple second videos corresponding to the storyboard audio.

3. The method according to claim 2, characterized in that Based on the audio duration and duration threshold of the storyboard audio, determining the number of videos to be extracted and the target duration includes: Determine the number of the plurality of first videos corresponding to the storyboard audio; Determine the number of videos as the number to be processed, determine whether the scaled duration of each of the first videos extracted according to the number to be processed and the audio duration is greater than or equal to the duration threshold, and if so, determine the number to be processed as the number of extracted videos, and determine the scaled duration combination of each of the extracted first videos as the target duration; Otherwise, the value of the video quantity minus one is determined as the new number to be processed and the process returns to continue judging until the number to be processed reaches one.

4. The method according to claim 2, characterized in that: The multiple third videos are subjected to time-length scaling processing according to the target time length to obtain multiple second videos corresponding to the storyboard audio, including: For each of the third videos, determine the comparison duration according to the zoomed duration of the third video corresponding to the target duration and the preset parameters, and determine whether the original duration of the third video is less than or equal to the comparison duration. If so, adjust the original duration of the third video to the zoomed duration by adjusting the playback speed to obtain the corresponding second video; Otherwise, a video segment of the zoomed duration is captured from the third video and determined as the corresponding second video.

5. The method according to claim 1, characterized in that Generating storyboard subtitles based on the storyboard audio and the storyboard text includes: Dividing the storyboard into multiple single sentences according to punctuation marks and character number thresholds; Determine the timestamp corresponding to each of the single sentences according to a uniform speed strategy based on the audio duration of the storyboard audio; The multiple single sentences and their corresponding multiple timestamp combinations are determined as corresponding storyboard subtitles.

6. The method according to claim 1, characterized in that After performing video screening and time-length scaling processing on the plurality of first videos according to the storyboard dimensions based on the storyboard audio to obtain a plurality of second videos, the method further includes: Determine a canvas size of a second video having the highest resolution and the largest number of aspect ratios among the plurality of second videos as a target canvas size; The video sizes of the plurality of second videos are scaled in multiple proportions according to the target canvas size.

7. The method according to claim 6, characterized in that Scaling the video sizes of the plurality of second videos in multiple proportions according to the target canvas size includes: Determining a video ratio of each of the second videos and a canvas ratio of the target canvas size; For each of the second videos, determine whether the video ratio of the second video is greater than the canvas ratio; if so, adjust the video width of the second video to the canvas width of the target canvas size, and adjust the video height according to the ratio of the canvas width to the video width; Otherwise, the video height of the second video is adjusted to the canvas height of the target canvas size, and the video width is adjusted according to the ratio of the canvas height to the video height.

8. A video generating device, characterized in that: include: An acquisition module, used for acquiring basic video data, wherein the basic video data includes a scene identifier, a target script, and a plurality of first videos; A storyboard module, used for generating storyboard text and storyboard audio based on the target script and the scene information corresponding to the scene identifier; A video editing module, configured to perform video screening and time-scaling processing on the plurality of first videos according to the storyboard dimensions based on the storyboard audio to obtain a plurality of second videos; A subtitle module, used for generating storyboard subtitles based on the storyboard audio and the storyboard copy; A video generation module is used to generate a target video based on the multiple second videos, the multiple storyboard audios and the multiple storyboard subtitles.

9. An electronic device, characterized in that: The electronic device comprises: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the video generation method described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and the computer program is used to execute the video generation method described in any one of claims 1 to 7.

Images