Video generation method and device, vehicle and storage medium

Abstract

The application discloses a video generation method and device, a vehicle and a storage medium, wherein the video generation method comprises the following steps: acquiring a required video theme and a video keyword feature; generating a plurality of segment prompt words based on the required video theme and the video keyword feature; generating a plurality of video segments based on the plurality of segment prompt words, wherein the plurality of segment prompt words correspond to the plurality of video segments one by one; and performing a clipping operation on the plurality of video segments to obtain a target video. The application solves the technical problem of high calculation complexity in the prior art when a video generation model automatically generates a video.

Description

Technical Field

[0001] This invention relates to the field of automation control technology, and more specifically, to a video generation method, apparatus, vehicle, and storage medium. Background Technology

[0002] Current technological advancements involve video generation within Artificial Intelligence Generated Content (AIGC), primarily based on a Diffusion Theory (DiT) architecture. This architecture achieves high-fidelity video synthesis by performing self-attention modeling on noise blocks in the latent space. This technology has been widely applied in scenarios such as film and television pre-production, advertising production, and social media content generation. It demonstrates significant potential to replace traditional manual production, especially in industrialized production environments with high visual quality requirements and diverse content creation needs.

[0003] However, existing technologies are limited by the inherent quadratic complexity of self-attention mechanisms. Their computational and memory overhead increases exponentially with the number of video frames and resolution, making it difficult to support long-sequence video generation in a single inference iteration. Mainstream models typically only output stable 10-20 second segments, failing to meet the automated production requirements of industrial-grade minute-long videos. The industry urgently needs a technological path that can overcome hardware bottlenecks and achieve continuous, stable, and highly consistent long-video generation. However, existing solutions, while maintaining generation quality, cannot effectively reduce the computational burden caused by sequence length. This means that long-video generation still relies on manual segmentation, splicing, and repeated parameter tuning, severely hindering automated production efficiency and the implementation of large-scale applications.

[0004] There is currently no effective solution to the above-mentioned technical problems. Summary of the Invention

[0005] This invention provides a video generation method, apparatus, vehicle, and storage medium to at least solve the technical problem of high computational complexity when automatically generating videos using existing video generation models.

[0006] According to one embodiment of the present invention, a video generation method is provided, comprising: obtaining a desired video theme and video keyword features; generating multiple segment prompts based on the desired video theme and video keyword features; generating multiple video segments based on the multiple segment prompts, wherein the multiple segment prompts correspond one-to-one with the multiple video segments; and performing editing operations on the multiple video segments to obtain a target video.

[0007] Optionally, the video generation method further includes: obtaining the target object's demand information; and determining the video theme and video keyword features based on the demand information.

[0008] Optionally, the video generation method further includes: generating video script based on the video theme and video keyword features; generating multiple script segments based on the video script; and generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0009] Optionally, the video generation method further includes: generating multiple initial video segments based on multiple segment prompts; calculating multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; constructing a video segment set based on the multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; regenerating the multiple video segments to be corrected to obtain multiple corrected video segments; and generating multiple video segments based on the multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0010] Optionally, the video generation method further includes: extracting at least one reference image from the requirements information; and generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0011] Optionally, the video generation method further includes: comparing multiple semantic fit scores with preset scores respectively to obtain comparison results; determining multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and constructing a video segment set based on the multiple video segments to be corrected.

[0012] Optionally, the video generation method further includes: performing editing operations on multiple video segments to obtain an initial video; obtaining the video rhythm of the initial video; determining the background music based on the video rhythm; and generating a target video based on the background music and the initial video.

[0013] Optionally, the video generation method further includes: arranging multiple video segments based on video text to obtain multiple arranged video segments; and performing editing operations on the multiple arranged video segments to obtain an initial video.

[0014] According to one embodiment of the present invention, a video generation apparatus is also provided, comprising: an acquisition module for acquiring a desired video theme and video keyword features; a first generation module for generating multiple segment prompts based on the desired video theme and video keyword features; a second generation module for generating multiple video segments based on the multiple segment prompts, wherein the multiple segment prompts correspond one-to-one with the multiple video segments; and an editing module for performing editing operations on the multiple video segments to obtain a target video.

[0015] Optionally, the acquisition module includes: a first acquisition unit for acquiring the demand information of the target object; and a first determination unit for determining the video theme and video keyword features based on the demand information.

[0016] Optionally, the first generation module includes: a first generation unit for generating video script based on the video theme and video keyword features; a second generation unit for generating multiple script segments based on the video script; and a third generation unit for generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0017] Optionally, the second generation module includes: a fourth generation unit, used to generate multiple initial video segments based on multiple segment prompts; a calculation unit, used to calculate multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; a first construction unit, used to construct a video segment set based on multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; a fifth generation unit, used to regenerate multiple video segments to be corrected to obtain multiple corrected video segments; and a sixth generation unit, used to generate multiple video segments based on multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0018] Optionally, the fourth generation unit includes: an extraction subunit for extracting at least one reference image from the requirement information; and a generation subunit for generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0019] Optionally, the construction unit includes: a comparison unit, used to compare multiple semantic fit scores with preset scores respectively to obtain comparison results; a second determination unit, used to determine multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and a second construction unit, used to construct a video segment set based on the multiple video segments to be corrected.

[0020] Optionally, the editing module includes: an editing unit for performing editing operations on multiple video segments to obtain an initial video; a second acquisition unit for acquiring the video rhythm of the initial video; a third determination unit for determining the background music of the video based on the video rhythm; and a seventh generation unit for generating a target video based on the background music and the initial video.

[0021] Optionally, the editing unit includes: an arrangement subunit for arranging multiple video segments based on the video text to obtain multiple arranged video segments; and an editing subunit for performing editing operations on the arranged multiple video segments to obtain the initial video.

[0022] According to one embodiment of the present invention, a vehicle is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the video generation method of any of the above claims.

[0023] According to one embodiment of the present invention, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the video generation method described in any of the preceding claims.

[0024] According to one embodiment of the present invention, a non-volatile storage medium is also provided, wherein a computer program is stored in the non-volatile storage medium, wherein the computer program is configured to execute the video generation method described in any of the above-mentioned embodiments when running.

[0025] According to one embodiment of the present invention, a computer program product is also provided, which stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the video generation method described above.

[0026] In this embodiment of the invention, by acquiring the required video theme and video keyword features, multiple segment prompt words are generated based on the required video theme and video keyword features, thereby achieving the purpose of generating multiple video segments based on multiple segment prompt words. The multiple segment prompt words correspond one-to-one with the multiple video segments, thereby achieving the technical effect of performing editing operations on multiple video segments to obtain the target video. This can solve the technical problem of high computational complexity when automatically generating videos in the existing technology. Attached Figure Description

[0027] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0028] Figure 1 This is a flowchart of a video generation method according to one embodiment of the present invention;

[0029] Figure 2 This is a flowchart of a method for obtaining video generation information according to one embodiment of the present invention;

[0030] Figure 3This is a flowchart of a method for generating video clip prompts according to one embodiment of the present invention;

[0031] Figure 4 This is a structural block diagram of a video generation apparatus according to one embodiment of the present invention;

[0032] Figure 5 This is a structural block diagram of an electronic device according to one embodiment of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate so that embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] According to an embodiment of the present invention, an embodiment of a video generation method is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system containing at least one set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0036] This method embodiment can also be executed in an electronic device, similar control device, or vehicle-mounted terminal that includes a memory and a processor. Taking a vehicle-mounted terminal as an example, the vehicle-mounted terminal may include one or more processors and a memory for storing data. Optionally, the vehicle-mounted terminal may also include a communication device for communication functions and a display device. Those skilled in the art will understand that the above structural description is merely illustrative and does not limit the structure of the vehicle-mounted terminal. For example, the vehicle-mounted terminal may include more or fewer components than those described above, or have a different configuration than those described above.

[0037] A processor may include one or more processing units. For example, a processor may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processing (DSP) chip, a microprocessor, a field-programmable gate array (FPGA), a neural network processing unit (NPU), a tensor processing unit (TPU), or an artificial intelligence (AI) type processor. Different processing units may be independent components or integrated into one or more processors. In some instances, electronic devices may also include one or more processors.

[0038] The memory can be used to store computer programs, such as the computer program corresponding to the video generation method in this embodiment of the invention. The processor implements the video generation method described above by running the computer program stored in the memory. The memory may include high-speed random access memory and non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include memory remotely located relative to the processor, and these remote memories can be connected to the electronic device via a grid. Examples of such grids include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0039] The communication device is used to receive or transmit data via a grid. Specific examples of the aforementioned grid may include a wireless grid provided by the mobile terminal's communication provider. In one example, the communication device includes a network interface controller (NIC), which can connect to other grid devices via a base station to communicate with the Internet. In another example, the communication device may be a radio frequency (RF) module used for wireless communication with the Internet. In some embodiments of this solution, the communication device is used to connect to mobile devices such as mobile phones and tablets, enabling the mobile device to send commands to the vehicle-mounted terminal.

[0040] The display device can be a touchscreen liquid crystal display (LCD) or a touch display (also referred to as a "touchscreen" or "touch display screen"). This LCD allows the user to interact with the user interface of the in-vehicle terminal. In some embodiments, the in-vehicle terminal has a graphical user interface (GUI), allowing the user to interact with the GUI through finger contact and / or gestures on a touch-sensitive surface. The human-machine interaction function may include a vehicle gear shifting function, and executable instructions for performing these functions are configured / stored in one or more processor-executable computer program products or readable storage media.

[0041] Figure 1 This is a flowchart of a video generation method according to one embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0042] Step S101: Obtain the required video theme and video keyword features.

[0043] Optionally, the execution subject in this embodiment is a video generation system. It should be noted that other electronic devices and processors can also be used as the execution subject, and no further limitations are made here.

[0044] In the technical solution provided by step S101 of the present invention, the user inputs a free text topic (e.g., "driverless traffic flow in the dawn of a future city"), and performs intent recognition and entity extraction through a pre-trained large language model (LLM) to extract core semantic elements, including subject (driverless vehicle), scene (future city), time (dawn), action (flow), and style keywords (technological feel, cool color tone, high dynamic range lighting effect).

[0045] Furthermore, the extracted semantic elements are mapped into multidimensional feature vectors in the vector space, and cross-modal encoders such as CLIP are used to transform text keywords into semantic embedding vectors aligned with the visual latent space, ensuring that they can be directly called by subsequent diffusion models.

[0046] Specifically, Large Language Model (LLM) is a natural language understanding model based on the Transformer architecture and pre-trained on massive amounts of text data, possessing the ability of contextual reasoning and semantic abstraction.

[0047] Specifically, the CLIP encoder is a cross-modal contrastive learning model proposed by OpenAI that can map text and images to the same semantic space, achieving text-visual semantic alignment.

[0048] As an optional implementation, the user inputs a natural language description through a text box, and the system calls the locally deployed LLM to perform word segmentation, dependency analysis and keyword extraction, and outputs structured JSON fields, including fields such as "topic", "scene tag" and "style constraint", which are then CLIP encoded to generate a 1024-dimensional semantic vector.

[0049] As another optional implementation, users can upload graphic scripts containing text descriptions (such as storyboard sketches with text descriptions). The system extracts the text content through OCR and inputs it into a multimodal encoder in conjunction with image visual features to extract composite keyword features that fuse text and visual cues, thereby enhancing the consistency between semantics and visuals.

[0050] It is worth noting that the above technical steps can transform unstructured user intent into structured semantic feature vectors that can be directly parsed by the video generation model, ensuring that the generation task has clear and quantifiable control basis at the semantic level, and improving the accuracy of semantic mapping from input instructions to visual output.

[0051] Step S102: Generate multiple segment prompts based on the required video theme and video keyword features.

[0052] In the technical solution provided by step S102 of the present invention, the input complete video theme is decomposed into several continuous time segments according to the preset duration constraint (such as 10 seconds per segment), and each segment corresponds to an independent narrative unit.

[0053] Furthermore, while maintaining global semantic consistency, a large language model (LLM) is used to generate independent but logically coherent prompts for each time segment. The prompts include elements such as scene description, main action, visual style and camera movement, and are injected with semantic constraints from the aforementioned video keyword features. Then, a time sequence number and context anchor identifier are added to each segment prompt to ensure that frame-level consistency control can be performed in the subsequent generation stage based on the order and semantic association.

[0054] The aforementioned prompts refer to structured text instructions used to guide the diffusion model in generating specific visual content, including scene, subject, action, style, and camera parameters.

[0055] As an optional implementation method, when generating documentary-style videos, the system divides the theme of "Dawn of the Future City" into three sub-segments: "First Light of Dawn", "Traffic System Start-up", and "Building Light and Shadow Gradually Brighten". LLM generates prompts containing specific time, perspective and dynamic elements, such as "In the morning mist, a convoy of driverless vehicles travels at a constant speed along the overpass, with cool-toned metallic reflections, wide-angle upward shot, and dynamic blur".

[0056] As another alternative implementation method, when generating advertising videos, the system generates differentiated prompts for the unified theme "smart living" based on brand style keywords (such as "technological" and "high saturation"). Each segment focuses on different product functions (such as starting the morning coffee machine and opening the smart curtains), maintaining a unified visual style but with separate content themes.

[0057] It is worth noting that decomposing a single long video topic into multiple independently generated short temporal cue word sequences ensures that each cue word meets the input requirements of the DiT model in terms of semantic integrity and stylistic consistency, while avoiding the surge in computational complexity caused by a single input of an extremely long sequence.

[0058] Step S103: Generate multiple video segments based on multiple segment prompts, wherein each segment prompt corresponds to one of the multiple video segments.

[0059] In the technical solution provided in step S103 of the present invention, each independent segment prompt word is used as an independent input and loaded into the text encoder of the underlying Diffusion Transformer (DiT) model to ensure that the semantic constraints accurately correspond to the generation target of a single video segment.

[0060] Furthermore, provided hardware resources permit, multiple prompts can be scheduled in parallel to different computing nodes to launch DiT models for iterative denoising of latent spatial noise, generating short video clips of corresponding durations. An independent but traceable random seed is then assigned to each clip generation process to support subsequent cross-segment feature alignment based on semantic anchors while maintaining consistency within the clip.

[0061] As an optional implementation, when generating a science fiction documentary, the system inputs the two prompts "driverless convoy driving along the viaduct in the morning light" and "glass curtain wall building complex gradually lights up" into two independent DiT inference instances, each generating a 10-second video clip. There is no shared computing state between the clips, and the style is unified only by relying on the visual keywords in the prompts (such as "cool color tone" and "metallic reflection").

[0062] As another alternative implementation, when generating brand advertisements, the system executes the prompt word generation task sequentially. Each segment uses the same model instance, but by resetting the initial noise in the latent space and loading the CLIP embedding vector of the corresponding prompt word, segment-by-segment generation is achieved, avoiding contextual interference.

[0063] The aforementioned Diffusion Transformer (DiT) is a diffusion model based on the Transformer architecture. It generates image or video frames by iteratively denoising in the latent space, which has the ability to preserve high detail. However, the computational complexity increases quadratically with the sequence length.

[0064] The aforementioned random seed is a numerical parameter that controls the initial noise distribution during the generation process. The same seed can reproduce the same generation result, while different seeds will produce different outputs.

[0065] It is worth noting that by breaking down the long video generation task into multiple low-complexity short sequence generation tasks, each segment can be generated with high quality under independent input conditions, effectively avoiding the memory explosion and computational delay problems caused by the self-attention mechanism when processing long sequences in a single run.

[0066] Step S104: Perform editing operations on multiple video segments to obtain the target video.

[0067] In the technical solution provided by step S104 of the present invention, all video segments that have passed the quality test are arranged in an orderly manner according to the time sequence number and semantic logical order carried by the pre-generated segment prompt words, so as to ensure narrative coherence.

[0068] Furthermore, automated editing tools are invoked to perform transition processing at the frame boundaries of adjacent video clips, including fade-in / fade-out, mask transitions, or motion matching, to eliminate visual jumps between clips. Then, based on the rhythm description in the original script, the beat and emotional tone of the background music (BGM) are automatically matched, and subtitle text is generated synchronously to achieve audio-visual timing alignment.

[0069] The aforementioned automated editing tools refer to the application programming interfaces (APIs) provided by open-source multimedia processing frameworks. These APIs support video encoding, decoding, transcoding, compositing, and audio-visual synchronization operations, and are widely used in automated video processing workflows.

[0070] As an optional implementation method, when generating corporate promotional videos, the system splices the three segments of "product debut", "user use" and "service response" in a logical order, uses a 0.5-second fade-in and fade-out transition, and selects 120BPM electronic music and dynamic subtitles to play synchronously based on the description of "fast-paced" in the script.

[0071] As another optional implementation, when generating a documentary, the system detects that the scene of an adjacent segment changes from daytime to nighttime, automatically inserts 5 frames of gradual darkening transition, and matches the ambient sound effects (such as transitioning from traffic sounds to insect chirps) to enhance the immersiveness of the scene transition.

[0072] It is worth noting that multiple independently generated short video clips are integrated into a continuous video in semantic order. Through automated transitions and audio-visual synchronization, the sense of discontinuity between clips is eliminated, forming a target video with a complete narrative structure.

[0073] Steps S101 to S104 above show that, in this invention, by acquiring the required video theme and video keyword features, multiple segment prompts are generated based on the required video theme and video keyword features, thus achieving the purpose of generating multiple video segments based on multiple segment prompts. Among them, multiple segment prompts correspond one-to-one with multiple video segments, thereby achieving the technical effect of performing editing operations on multiple video segments to obtain the target video. This can solve the technical problem of high computational complexity when automatically generating videos in the existing technology.

[0074] The method described in this embodiment will now be described in further detail.

[0075] Step S201: Obtain the target object's requirement information;

[0076] Step S202: Determine the video theme and video keyword features based on the demand information.

[0077] In this embodiment, such as Figure 2 As shown, it receives raw input from the user or system, which can be unstructured natural language text, speech-to-text, or structured form fields, covering video intent, style preferences, or functional goals.

[0078] Furthermore, the natural language processing module performs intent recognition, entity extraction, and keyword extraction on the collected demand information, transforming ambiguous semantics into clear video theme descriptions and a set of semantic keyword features that can be parsed by the visual model.

[0079] Specifically, the natural language processing module is a text understanding component built on pre-trained language models (such as BERT and Llama), which has the capabilities of syntactic analysis, semantic normalization, and key information localization.

[0080] Specifically, keyword features are semantic units extracted from text that have a visual guiding role. They are used to constrain the output style, scene elements, and compositional tendencies of the generative model and are usually expressed in the form of structured vectors or label sets.

[0081] As an optional implementation method, users can input "create a futuristic advertisement showcasing new energy vehicles driving under the aurora" through the interactive interface. The system will then extract the video theme as "new energy vehicles driving under the aurora" through word segmentation and semantic role labeling, with keyword features including "futuristic feel", "aurora background", "dynamic light and shadow", "low saturation cool color tone" and "high-speed shot".

[0082] As another optional implementation, the enterprise's marketing department uploads a document containing text and images. The system calls OCR to extract the text content and combines it with visual elements in the images (such as product shape and main color scheme) through a multimodal encoder for joint analysis, outputting a video with the theme "Smart Home Appliances and Family Scene Integration" and the keyword features "warm family atmosphere", "warm lighting", "human-computer interaction gestures", "minimalist design", and "slow-motion close-up".

[0083] It is worth noting that transforming diverse raw demand inputs into structured, computable video topic and keyword features provides a clear, consistent, and model-understandable semantic control basis for subsequent video generation.

[0084] Step S301: Generate video script based on the video theme and video keyword features;

[0085] Step S302: Generate multiple script segments based on the video text;

[0086] Step S303: Generate multiple script prompts based on multiple script fragments, wherein each script fragment corresponds to one of the multiple script prompts.

[0087] In this embodiment, such as Figure 3 As shown, the Large Language Model (LLM) is used to take the video's theme and keyword features as input to generate a coherent and complete narrative text, including the overall structure, emotional context, and sequence of key events. Based on preset duration constraints (e.g., 8–12 seconds per segment) and narrative rhythm, the video script is divided into several semantically complete and logically independent sub-units. Each sub-unit constitutes a script segment, clearly defining the start and end times and core visual content.

[0088] Furthermore, structured prompts are generated independently for each script fragment, covering scene descriptions, main actions, camera movements, lighting styles, and visual keywords, ensuring that they can be directly parsed by the diffusion model to generate matching visual content.

[0089] The aforementioned snippet prompts refer to structured text instructions designed specifically for diffusion models. They contain visual elements, style parameters, and shot descriptions to guide the generation process of video frames, possessing semantic accuracy and model executability.

[0090] As an optional implementation method, the system inputs the theme "Daily Life at a Polar Research Station" and keywords "coldness," "rigorous scientific research," and "loneliness." The system generates a 3-minute narrative script, which is then broken down into five script segments based on five time points: "Early Morning Equipment Inspection," "Laboratory Data Acquisition," "Aurora Passing By Outside the Window," "Communication Interruption," and "Nighttime Log Recording." Each segment is then converted into prompt words, such as "In the cold environment, researchers operate instruments while wearing heavy protective suits. The warm yellow indoor lighting contrasts with the deep blue aurora outside the window, creating a cool and warm contrast. Fixed mid-range shot, no motion blur."

[0091] As another optional implementation method, input the theme "unboxing video of children's educational toys" and the keywords "childlike fun", "bright colors" and "surprise", and the system will generate five script segments including "unboxing", "toy unveiling", "interactive demonstration", "laughter reaction" and "packaging recycling". Each segment generates corresponding prompt words, such as "a colorful plastic box is torn open by a child, the toy suddenly pops out, slow motion close-up, background is a pink room, direct sunlight, high saturation, cartoon style rendering".

[0092] It is worth noting that the macro-level video theme and style constraints are transformed into a set of independent prompt word sequences with clear structure, reasonable timing, and precise semantics, providing an executable and reproducible visual control basis for the segment-by-segment generation of subsequent video clips.

[0093] Step S401: Generate multiple initial video segments based on multiple segment prompts;

[0094] Step S402: Calculate multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts in multiple initial video segments, wherein each initial video segment corresponds to one semantic fit score;

[0095] Step S403: Construct a video segment set based on multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected;

[0096] Step S404: Regenerate multiple video segments to be corrected to obtain multiple corrected video segments;

[0097] Step S405: Generate multiple video segments based on multiple corrected video segments and target video segments, wherein the target video segments are the video segments other than the multiple video segments to be corrected among the multiple initial video segments.

[0098] In this embodiment, each segment cue word is independently input into the Diffusion Transformer (DiT) model to generate initial video segments of corresponding durations, with each segment output in parallel or serially. Then, a multimodal encoder such as CLIP is used to extract the visual features of each initial video segment and the textual features of the corresponding segment cue word, calculating their cosine similarity in the embedding space to generate a single semantic fit score.

[0099] Furthermore, initial video segments with scores below the threshold are marked as "video segments to be corrected", while the rest are classified as "target video segments". For each video segment to be corrected, the random seed is changed and the DiT model is called again to generate a new version. All corrected video segments are merged with unmarked target video segments to form the final set of multiple video segments.

[0100] The semantic fit score mentioned above is a numerical similarity between the visual features of the text and the video calculated by a multimodal model (such as CLIP), used to measure the degree of semantic matching between the generated content and the prompt words.

[0101] As an optional implementation, when generating the "Future City Transportation" promotional video, the system generates the 7th segment, "Aerial Cable Car Through Neon Buildings," with a semantic fit score of 0.52, which is lower than the threshold of 0.7. Due to the distorted shape of the cable car and the mismatch in the building style, the system triggers regeneration, and after changing the seed, outputs a corrected version with a fit score of 0.89. The scores of the remaining 14 segments are all greater than 0.7, and are directly retained as the target video segments.

[0102] As another optional implementation, when generating educational short videos, the system generates a "cell division process" segment. Due to dynamic blurring, the chromosome boundaries are blurred, and the CLIP score is only 0.61. The system automatically re-encodes the segment, using more precise anatomical keywords to enhance the prompts, and the final score is improved to 0.82. No intervention is required for the other segments.

[0103] It is worth noting that by using an automated semantic evaluation and closed-loop regeneration mechanism, video clips that do not meet the prompt word requirements are filtered and corrected, improving the consistency between the generated results and the semantic input, and forming a high-quality, high-accuracy set of video clips.

[0104] Step S501: Extract at least one reference image from the requirements information;

[0105] Step S502: Generate multiple initial video clips based on at least one reference image and multiple clip prompts.

[0106] In this embodiment, one or more reference images with style, composition, subject, or color features are automatically identified and extracted from the user's input requirements (such as text descriptions, uploaded images, historical material libraries, or related documents) through image recognition and semantic alignment technology. The extracted reference images are used as visual control signals and are input into the Diffusion Transformer (DiT) model along with the corresponding segment prompts. This guides the model to maintain consistency with the reference images in terms of subject shape, material texture, lighting style, or scene layout when generating each initial video segment.

[0107] The reference images mentioned above are static visual samples provided by the user or automatically identified by the system from the input. They are used to constrain the style, structure, or content of the generated video and serve as conditional inputs during the generation process.

[0108] The aforementioned Diffusion Transformer (DiT) is a diffusion model based on the Transformer architecture. It can receive textual cues and visual conditions (such as reference images) as joint inputs and generate a sequence of video frames that conform to semantic and style constraints through iterative denoising.

[0109] As an optional implementation, users upload a "cyberpunk-style photo of a futuristic city at night" as a reference and enter the theme "autonomous taxis traversing a neon-lit city". The system extracts visual features such as the main color tone (purple-blue gradient), building outlines (tall vertical structures), and light source characteristics (illuminated billboards and headlight trails) from the reference image. Combined with the clip prompt "a streamlined taxi speeds through the rain, raindrops reflect neon light spots, and the background is a holographic advertising screen", the system generates an initial video clip, ensuring that the vehicle design and lighting effects are highly consistent with the reference image.

[0110] As another optional implementation method, when generating educational animations, the system automatically extracts the cover image from the "children's picture book illustrations" provided by the user, extracts the cartoonish character shapes, rounded lines and soft color block features, uses it as a reference image, and combines it with the prompt "a rabbit wearing overalls is collecting glowing mushrooms in the forest" to generate an initial video clip that conforms to the style of the picture book, avoiding the generation of realistic animal images.

[0111] It is worth noting that by introducing reference images as visual control signals, the generated initial video clips can maintain consistency with the specified reference images in terms of main appearance, style characteristics, and scene composition, thereby improving the visual coherence of the generated content and the fidelity to the user's intent.

[0112] Step S601: Compare the multiple semantic fit scores with the preset scores respectively to obtain the comparison results;

[0113] Step S602: Based on the comparison results, multiple video segments to be corrected are identified, wherein the multiple video segments to be corrected are video segments whose semantic fit scores are lower than the preset scores;

[0114] Step S603: Construct a video clip collection based on multiple video clips to be corrected.

[0115] In this embodiment, the semantic fit score calculated for each initial video segment and the corresponding segment prompt word is compared with a preset threshold standard to determine whether it meets the minimum requirements for content fidelity.

[0116] Furthermore, based on the comparison results, all video segments with semantic fit scores below a preset threshold are automatically selected and marked as "video segments to be corrected". All video segments identified as needing correction are then aggregated to form an independent set of video segments, which serves as the input for the subsequent regeneration process.

[0117] Optionally, a threshold standard configured in the preset scoring test system is used to quantitatively determine whether a video segment meets the semantic consistency requirements. This standard is preset based on model performance, application scenario accuracy requirements, or user preferences.

[0118] Specifically, the video clip set is a collection of initial video clips with semantic fit below a threshold, which serves as the input source for the task to be regenerated.

[0119] As an optional implementation, when generating product promotional videos, the preset semantic fit score threshold is 0.85 (normalized matching score based on the CLIP model). The system scores 30 initial segments and finds that the 7th segment (“drone flying over factory roof”) has a score of 0.72. Because the shape of the drone in the picture does not match the description of “octagonal industrial-grade model”, it is identified as a segment to be corrected and added to the set to be corrected. The remaining 29 segments all have a score of 0.85 or higher and are not included in the set.

[0120] It is worth noting that, based on the automatic comparison between the semantic fit score and the preset threshold, video segments that do not meet the semantic requirements are accurately identified and categorized, forming a set of corrections that can be regenerated in a targeted manner, thereby achieving automated screening and isolation of low-quality outputs.

[0121] Step S701: Perform editing operations on multiple video segments to obtain the initial video;

[0122] Step S702: Obtain the video rhythm of the initial video;

[0123] Step S703: Determine the background music for the video based on the video rhythm;

[0124] Step S704: Generate the target video based on the background music and the initial video.

[0125] In this embodiment, multiple video segments that have passed quality verification are seamlessly spliced ​​together in semantic order to complete temporal connection and transition processing, forming an initial video that is played continuously. By analyzing the inter-frame motion change rate, shot switching frequency, subject movement speed and visual entropy fluctuation in the initial video, its temporal rhythm characteristics are calculated to form a rhythm curve in units of frame rate.

[0126] Furthermore, based on the extracted rhythm curve, audio segments with similar beat distribution, emotional curves, and dynamic range are retrieved from the preset music library to achieve automatic alignment between the background music and the video rhythm. The selected video background music is aligned with the initial video on the timeline, and the final target video is generated through audio fade-in / fade-out, audio-visual event triggering (such as matching drum beats with camera transitions).

[0127] Specifically, video rhythm is a temporal intensity index calculated by comprehensively considering visual dynamic features such as changes in optical flow between video frames, shot switching intervals, and subject movement speed. It is used to quantify the visual rhythm of a video.

[0128] As an optional implementation, when generating trailers for technology product launches, the initial video consists of multiple fast-paced shots (switching every 0.8 seconds per frame). The rhythm analysis module identifies an average of 1.5 visual abrupt changes and high-frequency motion trajectories per second. The system matches a track with a BPM of 128, featuring strong drum beats and electronic synthesizer pulses from the music library, and aligns the shot switching points with the drum beats to generate an impactful target video.

[0129] As another optional implementation method, when generating parent-child short videos, the initial video contains slow push-pull shots and gentle children's movements (low average inter-frame variation). Rhythm analysis identifies low-frequency fluctuations and soothing timing. The system selects a gentle piece with a BPM of 72, mainly featuring piano and strings, aligns the climax of the music with the close-up of the child's smile, and adds natural ambient sounds (birdsong, wind sounds) at the transitions to enhance the immersive experience, thus forming the target video.

[0130] It is worth noting that background music with time synchronization and emotional consistency is automatically matched and synthesized based on the visual dynamic characteristics of the initial video, thereby achieving automatic coordination of the rhythm of sound and image and enhancing the viewing smoothness and emotional expressiveness of the target video.

[0131] Step S801: Arrange multiple video clips based on the video text to obtain multiple arranged video clips;

[0132] Step S802: Perform editing operations on the arranged video segments to obtain the initial video.

[0133] In this embodiment, based on the explicit time sequence, logical relationship and narrative structure (such as "beginning - development - climax - ending") in the video script, a unique timestamp and location instruction are assigned to each video segment that has passed the quality test, ensuring that its arrangement is completely consistent with the event progression described in the script.

[0134] Furthermore, the automated editing engine is invoked to perform frame-level alignment and transition processing on adjacent video segments based on the sorting results, eliminating screen jumps and forming an initial video that plays continuously. This process does not include audio-visual enhancement or background music synchronization; it only completes the splicing of the visual sequence.

[0135] Specifically, video scripts are structured narrative texts generated by a scripting agent, containing event sequences, temporal relationships, and logical hierarchies, serving as the semantic basis for ordering video clips.

[0136] The above editing operation refers to the technical process of using automated tools to sequentially arrange, align frames, and process visual transitions of multiple video clips to achieve continuous playback, without involving audio addition or style enhancement.

[0137] As an optional implementation, the video script describes "morning sunlight → factory production line starts → workers wearing safety equipment enter the site → robotic arm performs precise operation → product rolls off the line". Based on this, the system arranges the five generated video clips in this order and inserts a 0.3-second cross-dissolve transition between adjacent clips to form an uninterrupted 12-second initial video.

[0138] As another optional implementation method, when generating a product launch documentary, the script clearly states "Act 1: R&D scene (laboratory) → Act 2: Manufacturing process (workshop) → Act 3: User use (home environment)". The system groups and sorts the corresponding video segments according to this three-level structure, inserts a 1-second black screen transition between each act, and finally outputs an initial video that conforms to the narrative act structure, without adding background music or subtitles.

[0139] It is worth noting that, based on the narrative structure defined by the video script, multiple independently generated video clips are arranged in a semantic logical order and seamlessly spliced ​​together to form an initial video with a coherent visual sequence.

[0140] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or grid device, etc.) to execute the methods of the various embodiments of the present invention.

[0141] This embodiment also provides a video generation apparatus for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0142] Figure 4 This is a structural block diagram of a video generation apparatus 400 according to one embodiment of the present invention, such as... Figure 4 As shown, the device includes: an acquisition module 41, a first generation module 42, a second generation module 43, and an editing module 44.

[0143] Module 41 is used to acquire the required video theme and video keyword features;

[0144] The first generation module 42 is used to generate multiple segment prompts based on the required video theme and video keyword features;

[0145] The second generation module 43 is used to generate multiple video segments based on multiple segment prompt words, wherein the multiple segment prompt words correspond one-to-one with the multiple video segments;

[0146] The editing module 44 is used to perform editing operations on multiple video clips to obtain the target video.

[0147] Optionally, the acquisition module 41 includes: a first acquisition unit for acquiring the demand information of the target object; and a first determination unit for determining the video theme and video keyword features based on the demand information.

[0148] Optionally, the first generation module 42 includes: a first generation unit for generating video script based on the video theme and video keyword features; a second generation unit for generating multiple script segments based on the video script; and a third generation unit for generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0149] Optionally, the second generation module 43 includes: a fourth generation unit, used to generate multiple initial video segments based on multiple segment prompts; a calculation unit, used to calculate multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; a first construction unit, used to construct a video segment set based on multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; a fifth generation unit, used to regenerate multiple video segments to be corrected to obtain multiple corrected video segments; and a sixth generation unit, used to generate multiple video segments based on multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0150] Optionally, the fourth generation unit includes: an extraction subunit for extracting at least one reference image from the requirement information; and a generation subunit for generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0151] Optionally, the construction unit includes: a comparison unit, used to compare multiple semantic fit scores with preset scores respectively to obtain comparison results; a second determination unit, used to determine multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and a second construction unit, used to construct a video segment set based on the multiple video segments to be corrected.

[0152] Optionally, the editing module 44 includes: an editing unit for performing editing operations on multiple video segments to obtain an initial video; a second acquisition unit for acquiring the video rhythm of the initial video; a third determination unit for determining the background music of the video based on the video rhythm; and a seventh generation unit for generating a target video based on the background music and the initial video.

[0153] Optionally, the editing unit includes: an arrangement subunit for arranging multiple video segments based on the video text to obtain multiple arranged video segments; and an editing subunit for performing editing operations on the arranged multiple video segments to obtain the initial video.

[0154] Embodiments of the present invention also provide a vehicle including a memory and a processor, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the video generation method described above.

[0155] Optionally, in this embodiment, the vehicle may be configured to store a computer program for performing the following steps:

[0156] Step S101: Obtain the required video theme and video keyword features;

[0157] Step S102: Generate multiple segment prompts based on the required video theme and video keyword features;

[0158] Step S103: Generate multiple video clips based on multiple clip prompts, wherein each clip prompt corresponds to one of the multiple video clips;

[0159] Step S104: Perform editing operations on multiple video segments to obtain the target video.

[0160] Optionally, the processor may also perform the following steps when executing the program: obtaining the target object's requirements information; and determining the video theme and video keyword features based on the requirements information.

[0161] Optionally, the processor may also perform the following steps when executing the program: generating video script based on the required video theme and video keyword features; generating multiple script segments based on the video script; and generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0162] Optionally, the processor, when executing the program, also implements the following steps: generating multiple initial video segments based on multiple segment prompts; calculating multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; constructing a video segment set based on the multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; regenerating the multiple video segments to be corrected to obtain multiple corrected video segments; generating multiple video segments based on the multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0163] Optionally, the processor also performs the following steps when executing the program: extracting at least one reference image from the requirements information; and generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0164] Optionally, when the processor executes the program, it also performs the following steps: comparing multiple semantic fit scores with preset scores respectively to obtain comparison results; determining multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and constructing a video segment set based on the multiple video segments to be corrected.

[0165] Optionally, the processor may also perform the following steps when executing the program: perform editing operations on multiple video segments to obtain an initial video; obtain the video rhythm of the initial video; determine the background music based on the video rhythm; and generate a target video based on the background music and the initial video.

[0166] Optionally, the processor may also perform the following steps when executing the program: arranging multiple video segments based on the video text to obtain multiple arranged video segments; and performing editing operations on the multiple arranged video segments to obtain the initial video.

[0167] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0168] Embodiments of the present invention also provide an electronic device, such as... Figure 5 As shown, it includes a memory 51 and a processor 52, wherein the memory stores a computer program and the processor is configured to run the computer program to perform the video generation method described above.

[0169] Optionally, in this embodiment, the electronic device may be configured to store a computer program for performing the following steps:

[0170] Step S101: Obtain the required video theme and video keyword features;

[0171] Step S102: Generate multiple segment prompts based on the required video theme and video keyword features;

[0172] Step S103: Generate multiple video clips based on multiple clip prompts, wherein each clip prompt corresponds to one of the multiple video clips;

[0173] Step S104: Perform editing operations on multiple video segments to obtain the target video.

[0174] Optionally, the processor may also perform the following steps when executing the program: obtaining the target object's requirements information; and determining the video theme and video keyword features based on the requirements information.

[0175] Optionally, the processor may also perform the following steps when executing the program: generating video script based on the required video theme and video keyword features; generating multiple script segments based on the video script; and generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0176] Optionally, the processor, when executing the program, also implements the following steps: generating multiple initial video segments based on multiple segment prompts; calculating multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; constructing a video segment set based on the multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; regenerating the multiple video segments to be corrected to obtain multiple corrected video segments; generating multiple video segments based on the multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0177] Optionally, the processor also performs the following steps when executing the program: extracting at least one reference image from the requirements information; and generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0178] Optionally, when the processor executes the program, it also performs the following steps: comparing multiple semantic fit scores with preset scores respectively to obtain comparison results; determining multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and constructing a video segment set based on the multiple video segments to be corrected.

[0179] Optionally, the processor may also perform the following steps when executing the program: perform editing operations on multiple video segments to obtain an initial video; obtain the video rhythm of the initial video; determine the background music based on the video rhythm; and generate a target video based on the background music and the initial video.

[0180] Optionally, the processor may also perform the following steps when executing the program: arranging multiple video segments based on the video text to obtain multiple arranged video segments; and performing editing operations on the multiple arranged video segments to obtain the initial video.

[0181] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0182] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program configured to execute the video generation method described above when run on a computer or processor.

[0183] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:

[0184] Step S101: Obtain the required video theme and video keyword features;

[0185] Step S102: Generate multiple segment prompts based on the required video theme and video keyword features;

[0186] Step S103: Generate multiple video clips based on multiple clip prompts, wherein each clip prompt corresponds to one of the multiple video clips;

[0187] Step S104: Perform editing operations on multiple video segments to obtain the target video.

[0188] Optionally, the storage medium is configured to store program code for performing the following steps: obtaining the target object's requirements information; and determining the video theme and video keyword features based on the requirements information.

[0189] Optionally, the storage medium is configured to store program code for performing the following steps: generating video script based on the characteristics of the required video theme and video keywords; generating multiple script segments based on the video script; and generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0190] Optionally, the storage medium is configured to store program code for performing the following steps: generating multiple initial video segments based on multiple segment prompts; calculating multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; constructing a video segment set based on the multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; regenerating the multiple video segments to be corrected to obtain multiple corrected video segments; generating multiple video segments based on the multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0191] Optionally, the storage medium is configured to store program code for performing the following steps: extracting at least one reference image from the requirements information; and generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0192] Optionally, the storage medium is configured to store program code for performing the following steps: comparing multiple semantic fit scores with preset scores respectively to obtain comparison results; determining multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and constructing a video segment set based on the multiple video segments to be corrected.

[0193] Optionally, the storage medium is configured to store program code for performing the following steps: performing editing operations on multiple video segments to obtain an initial video; obtaining the video rhythm of the initial video; determining video background music based on the video rhythm; and generating a target video based on the video background music and the initial video.

[0194] Optionally, the storage medium is configured to store program code for performing the following steps: arranging multiple video segments based on the video text to obtain multiple arranged video segments; and performing editing operations on the multiple arranged video segments to obtain an initial video.

[0195] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0196] Embodiments of the present invention also provide a computer program product, including a computer program, wherein the computer program, when executed by a processor, implements the steps of the video generation method described above.

[0197] Optionally, in this embodiment, the computer program product described above may be configured to store a computer program for performing the following steps:

[0198] Step S101: Obtain the required video theme and video keyword features;

[0199] Step S102: Generate multiple segment prompts based on the required video theme and video keyword features;

[0200] Step S103: Generate multiple video clips based on multiple clip prompts, wherein each clip prompt corresponds to one of the multiple video clips;

[0201] Step S104: Perform editing operations on multiple video segments to obtain the target video.

[0202] Optionally, when the computer program executes the program, it also performs the following steps: obtaining the target object's requirement information; and determining the video theme and video keyword features based on the requirement information.

[0203] Optionally, when the computer program executes the program, it also performs the following steps: generating video script based on the video theme and video keyword features; generating multiple script segments based on the video script; generating multiple segment prompts based on the multiple script segments, wherein the multiple script segments correspond one-to-one with the multiple segment prompts.

[0204] Optionally, when the computer program executes the program, it further implements the following steps: generating multiple initial video segments based on multiple segment prompts; calculating multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; constructing a video segment set based on the multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; regenerating the multiple video segments to be corrected to obtain multiple corrected video segments; generating multiple video segments based on the multiple corrected video segments and a target video segment, wherein the target video segment is the video segment other than the multiple video segments to be corrected among the multiple initial video segments.

[0205] Optionally, when the computer program executes the program, it also performs the following steps: extracting at least one reference image from the requirements information; and generating multiple initial video clips based on the at least one reference image and multiple clip prompts.

[0206] Optionally, when the computer program executes the program, it also performs the following steps: comparing multiple semantic fit scores with preset scores respectively to obtain comparison results; determining multiple video segments to be corrected based on the comparison results, wherein the multiple video segments to be corrected are video segments with semantic fit scores lower than preset scores; and constructing a video segment set based on the multiple video segments to be corrected.

[0207] Optionally, when the computer program executes the program, it also performs the following steps: performing editing operations on multiple video segments to obtain an initial video; obtaining the video rhythm of the initial video; determining the background music for the video based on the video rhythm; and generating a target video based on the background music and the initial video.

[0208] Optionally, when the computer program executes the program, it also performs the following steps: arranging multiple video segments based on the video text to obtain multiple arranged video segments; and performing editing operations on the multiple arranged video segments to obtain the initial video.

[0209] Optionally, specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementations, and will not be repeated here.

[0210] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0211] In the embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some interfaces; indirect couplings or communication connections between units or modules may be electrical or other forms.

[0212] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0213] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0214] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or grid device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0215] The above are merely preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for generating video, characterized in that, include: Obtain the required video themes and keyword features; Based on the video theme and video keyword features, generate multiple segment prompts; Multiple video clips are generated based on the multiple segment prompt words, wherein each of the multiple segment prompt words corresponds one-to-one with the multiple video clips; Editing operations are performed on the multiple video segments to obtain the target video.

2. The video generation method according to claim 1, characterized in that, Obtaining the required video topic and the video keyword features includes: Obtain the target object's requirements information; The video theme and video keyword features are determined based on the aforementioned demand information.

3. The video generation method according to claim 1, characterized in that, Based on the required video theme and the video keyword features, the multiple segment prompts are generated, including: Generate video script based on the video theme and video keyword features as described in the requirements; Multiple script fragments are generated based on the video text; The plurality of script fragments are used to generate the plurality of fragment prompt words, wherein the plurality of script fragments and the plurality of fragment prompt words correspond one-to-one.

4. The video generation method according to claim 2, characterized in that, Generating the multiple video segments based on the multiple segment prompts includes: Multiple initial video segments are generated based on the multiple segment prompts; Calculate multiple semantic fit scores for each initial video segment and its corresponding multiple segment prompts, wherein each initial video segment corresponds to one semantic fit score; A video segment set is constructed based on the multiple semantic fit scores, wherein the video segment set includes multiple video segments to be corrected; The multiple video segments to be corrected are regenerated to obtain multiple corrected video segments; The plurality of video segments are generated based on the plurality of corrected video segments and the target video segment, wherein the target video segment is a video segment other than the plurality of corrected video segments among the plurality of initial video segments.

5. The video generation method according to claim 4, characterized in that, Generating the multiple initial video segments based on the multiple segment cue words includes: Extract at least one reference image from the aforementioned requirements information; The plurality of initial video clips are generated based on the at least one reference image and the plurality of segment prompts.

6. The video generation method according to claim 4, characterized in that, Constructing the video segment set based on the multiple semantic fit scores includes: The multiple semantic fit scores are compared with preset scores to obtain comparison results; Based on the comparison results, the plurality of video segments to be corrected are determined, wherein the plurality of video segments to be corrected are video segments whose semantic fit scores are lower than the preset scores; The video segment set is constructed based on the multiple video segments to be corrected.

7. The video generation method according to claim 3, characterized in that, Performing the editing operation on the plurality of video segments to obtain the target video includes: The editing operation is performed on the multiple video segments to obtain the initial video; Obtain the video rhythm of the initial video; The background music for the video is determined based on the video rhythm; The target video is generated based on the background music and the initial video.

8. The video generation method according to claim 7, characterized in that, Performing the editing operation on the plurality of video segments to obtain the initial video includes: The multiple video segments are arranged based on the video text to obtain multiple arranged video segments; The editing operation is performed on the arranged video segments to obtain the initial video.

9. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to run the computer program to perform the video generation method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program is configured to execute the video generation method according to any one of claims 1 to 7 when run on a computer or processor.

Images