Short video generation method, device and equipment, storage medium and program product

Abstract

This disclosure provides a method, apparatus, device, storage medium, and program product for generating short drama videos, relating to artificial intelligence technologies such as large language models, intelligent agents, director's script generation, and video generation. The method includes: determining the story content and corresponding plot shooting type of each act within an episode script; determining a director's script design template and a director's script content template matching the plot shooting type; generating an actual director's script design using the story content of each act and the matching director's script design template, and generating a complete director's script containing the actual director's script content corresponding to each act using the actual director's script design and the matching director's script content template; and generating an episode-specific short video corresponding to the episode script based on the complete director's script for each act and a preset multi-layered cascading video generation mechanism. This solution achieves efficient production of short drama videos from episode scripts through the use of structured templates and a layered generation mechanism.

Description

Technical Field

[0001] This disclosure relates to the field of data processing technology, specifically to artificial intelligence technologies such as large language models, intelligent agents, director's script generation, and video generation, and particularly to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating short drama videos. Background Technology

[0002] With the development of Artificial Intelligence Generated Content (AIGC) technology, AI is being applied more and more widely in the field of film and television content creation.

[0003] In the field of short dramas and video generation, the director's script design and video generation process remain key bottlenecks in achieving automated production of high-quality film and television content. Summary of the Invention

[0004] This disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for generating short drama videos.

[0005] In a first aspect, this disclosure proposes a method for generating short drama videos, comprising: determining the story content and corresponding plot shooting type of each scene in an episode script; determining a director's script design template and a director's script content template that match the plot shooting type, pre-recording the correspondence between different plot shooting types and different director's script design templates and different director's script content templates, wherein the director's script design template and the director's script content template are obtained by templated processing of the director's scripts of historical film and television works with actual scores exceeding a preset score; generating an actual director's script design using the story content of each scene and the matching director's script design template, and generating a complete director's script containing the actual director's script content corresponding to each scene using the actual director's script design and the matching director's script content template; and generating an episode short video corresponding to the episode script based on the complete director's script of each scene and a preset multi-level cascading video generation mechanism.

[0006] Secondly, this disclosure proposes a short drama video generation device, comprising: an episode script analysis unit configured to determine the story content and corresponding plot shooting type of each scene in the episode script; a template matching unit configured to determine a director script design template and a director script content template matching the plot shooting type, wherein the correspondence between different plot shooting types and different director script design templates and different director script content templates is pre-recorded, and the director script design template and the director script content template are obtained by template processing of the director scripts of historical film and television works with actual scores exceeding preset scores; a complete director script generation unit configured to generate an actual director script design using the story content of each scene and the matched director script design template, and to generate a complete director script containing the actual director script content corresponding to each scene using the actual director script design and the matched director script content template; and a multi-layer cascaded video generation unit configured to generate episode short videos corresponding to the episode script based on the complete director script of each scene and a preset multi-layer cascaded video generation mechanism.

[0007] Thirdly, embodiments of this disclosure provide an electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to implement the short drama video generation method described in the first aspect.

[0008] Fourthly, embodiments of this disclosure provide a non-transitory computer-readable storage medium storing computer instructions that enable a computer to implement the short drama video generation method as described in the first aspect when executed.

[0009] Fifthly, embodiments of this disclosure provide a computer program product including a computer program that, when executed by a processor, can implement the steps of the short drama video generation method as described in the first aspect.

[0010] The short drama video generation solution disclosed herein firstly matches a determined plot shooting type with director's script design templates and director's script content templates derived from highly-rated historical films and television works. This transforms professional directing experience into reusable parameters, ensuring that the camera language conforms to industry standards. Secondly, it uses templates to generate actual director's script designs and fills in script content, addressing issues such as fragmented camera language and uncontrolled pacing in the generated videos. Finally, a multi-layered cascading video generation mechanism enables layer-by-layer quality control, ensuring character consistency, action coherence, and audio-visual synchronization. This solution achieves efficient short drama video production through the use of structured templates and a layered generation mechanism.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] Other features, objects, and advantages of this disclosure will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0013] Figure 1 This is an exemplary system architecture to which this disclosure can be applied;

[0014] Figure 2 A flowchart of a short drama video generation method provided in this disclosure embodiment;

[0015] Figure 3 A flowchart illustrating a method for generating and filtering multiple alternative complete director's scripts for a specific scene, provided in this embodiment of the disclosure;

[0016] Figure 4 A flowchart illustrating a method for guiding modifications based on correctness evaluation, provided as an embodiment of this disclosure;

[0017] Figure 5 A flowchart of a multi-layer cascaded video generation method provided in this disclosure embodiment;

[0018] Figure 6 A flowchart illustrating a complete short drama video generation method provided in this embodiment of the disclosure;

[0019] Figure 7 A structural block diagram of a short drama video generation device provided in this disclosure embodiment;

[0020] Figure 8 This is a schematic diagram of the structure of an electronic device suitable for performing a short drama video generation method, provided as an embodiment of the present disclosure. Detailed Implementation

[0021] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding; these should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description. It should be noted that, unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0022] The collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0023] Figure 1 An exemplary system architecture 100 is shown, to which embodiments of the short video generation methods, apparatuses, electronic devices, and computer-readable storage media of this disclosure can be applied.

[0024] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0025] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various applications for enabling information communication between the terminal devices 101, 102, and 103 and server 105 can be installed. These applications include short drama generation applications, episode-based short video generation applications, and instant messaging applications.

[0026] Terminal devices 101, 102, and 103 and server 105 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays, including but not limited to smartphones, tablets, laptops, and desktop computers. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices, and can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here. When server 105 is hardware, it can be implemented as a distributed server cluster composed of multiple servers, or as a single server. When server 105 is software, it can be implemented as multiple software programs or software modules, or as a single software program or software module; no specific limitation is made here.

[0027] Server 105 can provide various services through its built-in applications. Taking an episode-based short video generation application as an example, which generates episode-based short videos based on input episode scripts, server 105 can achieve the following effects when running this application: First, it receives episode scripts from terminal devices 101, 102, and 103 via network 104 and determines the story content and corresponding shooting type of each scene in the episode script; then, it determines the director's script design template and director's script content template matching the shooting type, and pre-records different shooting types and different director's script design templates. The correspondence between different director's script content templates is established. The director's script design template and director's script content template are obtained by templated processing of director's scripts of historical film and television works with actual scores exceeding the preset score. Next, the actual director's script design is generated using the story content of each scene and the matching director's script design template. Then, the complete director's script containing the actual director's script content corresponding to each scene is generated using the actual director's script design and the matching director's script content template. Finally, based on the complete director's script of each scene and the preset multi-level cascading video generation mechanism, the episode short video corresponding to the episode script is generated.

[0028] It should be noted that, in addition to being temporarily obtained from terminal devices 101, 102, and 103 via network 104, episode scripts can also be pre-stored locally on server 105 through various means. Therefore, when server 105 detects that this data is already stored locally (e.g., when starting to process previously reserved episode short video generation tasks), it can choose to directly obtain this data from locally. In this case, the exemplary system architecture 100 may also exclude terminal devices 101, 102, and 103 and network 104.

[0029] Since generating videos based on scripts requires significant computing resources and power, the short drama video generation methods provided in the subsequent embodiments of this disclosure are generally executed by a server 105 with strong computing power and abundant computing resources. Correspondingly, the short drama video generation device is also generally located within the server 105. However, it should also be noted that when terminal devices 101, 102, and 103 also possess sufficient computing power and resources, they can also perform the aforementioned calculations performed by the server 105 through their installed episode-based short video generation applications, thereby outputting the same results as the server 105. Especially when multiple terminal devices with different computing capabilities exist simultaneously, but the episode-based short video generation application determines that the terminal device has strong computing power and abundant remaining computing resources, it can allow the terminal device to perform the aforementioned calculations, thereby appropriately reducing the computing pressure on the server 105. Accordingly, the short drama video generation device can also be located within terminal devices 101, 102, and 103. In this case, the exemplary system architecture 100 may also exclude the server 105 and the network 104.

[0030] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0031] Please refer to Figure 2 , Figure 2 A flowchart of a short drama video generation method provided in this disclosure embodiment, wherein process 200 includes the following steps:

[0032] Step 201: Determine the story content of each scene in the episode script and the corresponding shooting type;

[0033] This step is intended for the execution body of the short drama video generation method (e.g., Figure 1 The server 105 shown determines the story content of each act in the episode script and the corresponding plot shooting type. For example, it can determine the plot shooting type of the episode script by semantically parsing the story content of each act in the episode script and by using type mapping technology. Here, semantic parsing refers to converting natural language text into a structured representation that can be understood by machines, and type mapping technology is to map information in words or sentences to specific types or categories.

[0034] Specifically, the story content of each act can be analyzed in the following way: First, the scene boundaries are identified by the spatiotemporal labeling algorithm, and the entities such as characters and props are located by named entity recognition to construct an entity relationship graph. Then, the action chain is analyzed by the event extraction model to form a "subject-action-object" triple structure. Finally, the conflict intensity is quantified by the sentiment analysis model, and the story content of each act containing spatiotemporal coordinates, character behavior chains and sentiment tags is output in the script.

[0035] Specifically, the shooting type of the episode script can be determined based on a multi-dimensional classification model. Standardized scene descriptions can be input into the multi-dimensional classification model, which will output matching items in the preset type system. In the multi-dimensional classification model, the classification criteria include conflict feature dimension (distinguishing between physical confrontation and verbal confrontation through action verb analysis), spatial enclosure dimension (judging between narrow spaces and open scenes based on scene descriptions, affecting the setting of camera movement range), and emotional intensity dimension (the shooting style is divided into soothing or intense types based on the level of emotional value).

[0036] Another specific implementation method is as follows: First, based on the script content of each act contained in the episode script, determine the actual plot that occurs in the corresponding scene of the corresponding act and the corresponding characters; then determine the shooting type that should be used to shoot the video content of the actual plot that occurs in the corresponding scene for each act, and obtain the plot shooting type corresponding to each act.

[0037] Furthermore, when a matching scene shooting type cannot be detected or determined, temporary type tags can be automatically created and their visual characteristics recorded. Cross-screen correlation analysis is also supported; if three consecutive scenes are of the same type, optimization suggestions are generated (such as "switch to an overhead shot in the third scene to avoid visual fatigue").

[0038] Step 202: Determine the director's script design template and director's script content template that match the type of story shooting;

[0039] Building upon step 201, this step aims to have the aforementioned implementing entity determine a director's script design template and a director's script content template that match the narrative shooting type. Both the director's script design template and the director's script content template are derived from templates generated from director's scripts of historical film and television works with historical ratings exceeding a preset threshold, sourced from the template library.

[0040] Specifically, the director's script design template may include at least one of the following template configuration items to be determined: scene structure, shot breakdown corresponding to each scene, temporal relationship between shots, and shooting techniques. The scene structure defines the overall narrative framework of the script unit, including the scene type (e.g., "beginning-development-climax-ending"), emotional intensity curve (e.g., the climax scene must contain a conflict value of 0.8 or higher), and time constraints (e.g., the development scene must account for 40% of the total duration). The shot breakdown corresponding to each scene allows for a refined decomposition of scene visual elements. Each scene can be broken down into executable shot units, containing three dimensions of parametric constraints: basic parameters (e.g., number of shots ≥ 5 / scene), type parameters (e.g., close-up shots ≥ 30%), and spatial parameters (e.g., it must contain at least one cross-axis shot). The temporal relationship between shots... Relationships are used to ensure the logic and rhythm of shot transitions. By modeling shot dependencies using a directed acyclic graph, three types of mandatory constraints can be defined: continuity constraints (e.g., close-up A must be followed by a wide shot B), time constraints (e.g., flashback shot duration ≤ 3 seconds), and causal constraints (e.g., the "drawing a gun" action must be preceded by a "loading a bullet" shot). The shooting techniques used can transform abstract director instructions into quantifiable parameters. Basic techniques can be defined through parameters (e.g., push / pull / pan / track), mechanical indicators can be set through parameters (e.g., movement speed 1.2m / s ± 0.3), and environmental variables can be bound through parameters (rainy weather shooting requires the lens waterproof marking to be turned on).

[0041] The director's script content template can preset multiple key parameters, such as shooting techniques, character dialogue, action logic, and video length. In practice, the storyboard requirements in the actual director's script design can be analyzed first, and the corresponding parameter groups can be extracted from the director's script content template. Then, placeholders can be replaced with specific values ​​through entity binding, and the speech rate can be adjusted according to the emotional intensity model. For action logic, the temporal rationality can be verified through a behavior rule library. For example, if it is detected that "a punch misses" but there is no subsequent action, the shot instruction "Character B counterattacks" can be automatically completed.

[0042] During template matching, the executing entity can use the plot shooting type as an index and calculate the template relevance based on multiple dimensions, including conflict intensity, spatial closure, and emotional tone, to determine the director's script design template and director's script content template that match the plot shooting type. For example, when a scene is classified as "ancient costume martial arts," the primary search will activate the action design template with the highest relevance in the basic template library; if it is detected that the current scene's emotional intensity deviates from the template's preset value by more than a preset threshold, then semantic enhancement search is initiated, dynamically replacing unsuitable modules through cosine similarity calculation.

[0043] Furthermore, when the rating of a newly released film or television work exceeds a threshold, an incremental training process can be automatically triggered to extract its innovative visual language and add it to the template library; when the audience is detected to be positioned in a specific regional market, a cultural adapter is activated to strengthen the regional aesthetic characteristics.

[0044] Step 203: Generate an actual director's script design using the story content of each act and the matching director's script design template, and generate a complete director's script containing the actual director's script content corresponding to each act using the actual director's script design and the matching director's script content template.

[0045] Building upon step 202, this step aims to enable the executing entity to generate an actual director's script design using the story content of each act and a matching director's script design template, thus transforming the script into executable director's instructions. Specifically, key event nodes for each act are first extracted from the story content. Then, an event importance analysis algorithm maps each key event node to the corresponding structural position in the director's script design template, binding entity parameters and inheriting the duration constraints of the template to generate the actual director's script design. The event importance analysis algorithm analyzes historical events or data using methods such as frequency, weighting, impact, and clustering to calculate the event's influence.

[0046] Specifically, the actual director's script design for each scene can be generated by using a director's script content template that matches the corresponding scene, resulting in a complete director's script for each scene. The actual script content includes at least one of the following: shooting techniques to be used in different shots, character dialogue, action logic, and video length. In other words, template matching is performed on a scene-by-scene basis.

[0047] In practice, the abstract instructions in the actual director's script design can be transformed into quantifiable parameters, character behavior chains can be constructed, and dialogue trigger nodes can be bound. Then, the content density can be optimized in reverse according to the length of the scene. If the dialogue is too long, redundant descriptions can be compressed and pause marks can be inserted to generate a complete director's script containing the content of the actual director's script for each scene.

[0048] Furthermore, when the actual director's script involves the collaboration of multiple elements, a timestamp mapping relationship can be established, and the video duration can be dynamically adjusted through a reverse constraint mechanism. For example, when the generated dialogue text exceeds the preset duration of the template, the pause interval will be automatically compressed or the action description will be simplified to ensure that the storyboard duration strictly matches the script design.

[0049] For non-linear narratives (such as flashbacks), multiple sets of spatiotemporal parameters (cool colors in reality / warm colors in memories) can be created and automatically activated during splicing; and a physical feasibility verification module can be introduced to automatically add a "green screen compositing" mark and corresponding special effects instructions when a "fall from a height" without safety measures is detected; a cross-screen parameter inheritance mechanism can also be established to track key visual elements (such as the color of the protagonist's coat), and if the color parameters are modified in subsequent scenes, correction instructions will be automatically added to the close-up of the front-facing camera.

[0050] To address discrepancies between the template and the design, an adaptive compensation process can be initiated: historical parameters from similar scenes can be retrieved to generate supplementary instructions; if no reference data is available, reasonable parameters can be calculated based on a physics simulator. Manual intervention is also supported, allowing directors to directly modify the generated script content using annotation tools and record adjustment points, feeding them back to the template library to create a closed loop of continuous optimization.

[0051] Step 204: Based on the complete director's script for each scene and the preset multi-layered cascading video generation mechanism, generate episode short videos corresponding to the episode scripts.

[0052] Building upon step 203, this step aims to generate episode-specific short videos corresponding to the episode scripts using the aforementioned execution entity based on the complete director's script for each scene and a pre-defined multi-layered cascaded video generation mechanism. The multi-layered cascaded video generation mechanism achieves high-quality conversion from director's scripts to episode-specific short videos through a layered and progressive synthesis strategy. Specifically, static visual anchor points are generated based on the storyboard descriptions in the director's script. A text-to-graphic agent parses text instructions into spatial composition parameters to output basic keyframes. A graph-to-graphic agent then performs cross-scene consistency optimization, identifying core elements through a feature extraction model and comparing them with a global parameter library. If a conflict is detected, feature mapping correction is automatically triggered, replacing abnormal elements while preserving the composition.

[0053] Specifically, an action interpolation engine can be used to calculate pixel displacement trajectories (such as arm waving paths) between the first and last keyframes based on optical flow. Redundant actions are compressed using a time warp algorithm, ensuring precise matching of single-scene duration to the script timecode. A strong correlation model between audio tracks and lip movements is established for the dialogue audio binding process: dialogue is decomposed into phoneme sequences to drive a 3D facial model, ensuring strict synchronization between lip and tooth movements at 30 frames per second and the audio waveform. An environmental sound field simulation module can also be introduced, calling acoustic databases to mix environmental sound effects (such as rain and footsteps) based on scene tags, and matching spatial acoustic characteristics (such as echo attenuation coefficients in narrow alleys) using a convolutional reverberation algorithm. Furthermore, inter-frame transition functions can be calculated based on the scene sequence relationship through scene-level splicing. Dynamic frame sampling technology is used for fast-cut scenes, inserting fade-in and fade-out at switching points to avoid visual jumps, and tracking key parameters through a global state recorder. When the executing entity detects scene continuity between adjacent scenes, it automatically inherits terminal lighting parameters to eliminate color banding.

[0054] The short drama video generation method provided in this disclosure firstly matches the determined plot shooting type with director's script design templates and director's script content templates derived from highly-rated historical films and television works, thereby transforming professional directing experience into reusable parameters and ensuring that the camera language conforms to industry standards. Secondly, it uses the templates to generate actual director's script designs and fill in the script content to solve the problems of fragmented camera language and uncontrolled pacing in the generated videos. Finally, a multi-layered cascaded video generation mechanism enables layer-by-layer quality control, ensuring character consistency, action coherence, and audio-visual synchronization. This solution achieves efficient short drama video production through the use of structured templates and a layered generation mechanism.

[0055] Given the inherent instability of videos generated using large language models, even with a relatively accurate complete director's script, there's no guarantee that the generated video will meet expectations. Therefore, to ensure the generated video closely approximates expectations and to better evaluate the generated complete director's script, please refer to [the relevant documentation / reference]. Figure 3 , Figure 3 A flowchart of a method for generating and filtering multiple alternative complete director's scripts for a specific scene, provided in this embodiment of the disclosure, wherein process 300 includes the following steps:

[0056] Step 301: For the target scenes corresponding to key plot points in multiple scenes, design the actual director's script for each target scene and multiple matching director's script content templates, and use a tree search algorithm to generate multiple alternative complete director's scripts corresponding to each target scene in parallel.

[0057] This step aims to have the aforementioned executing entity, for target scenes corresponding to key plot points across multiple scenes, utilize the actual director's script design for each target scene and multiple matching director's script content templates, and then use a tree search algorithm in parallel to generate multiple alternative complete director's scripts corresponding to each target scene. The tree search algorithm is an algorithm used for searching within graphical or tree structures.

[0058] In this embodiment, the executing entity first identifies target scenes in the script with high conflict or emotional intensity exceeding a threshold. The actual director's script design for each target scene is combined with multiple matching content templates to create an independent search tree for each target scene. The root node is the original script design, and each branch represents a content template filling scheme.

[0059] Specifically, the implementing entity can identify key scenes through a plot importance analysis algorithm. The judgment criteria include three dimensions: first, the emotional intensity value (e.g., the emotional value of a conflict scene is ≥0.8); second, the narrative node weight (e.g., "twist" and "climax" are marked as key events); and third, the character participation density (scenes where the main character's screen time accounts for more than 70%).

[0060] Step 302: For the target scenes corresponding to key plot points in multiple scenes, generate pending storyboard videos corresponding to each of the multiple alternative complete director scripts for each target scene according to the multi-level cascaded video generation mechanism. Determine the target storyboard video based on the actual effect evaluation of each pending storyboard video, and cut off all alternative complete director scripts that do not correspond to the target storyboard video.

[0061] For each target scene within a multi-scene framework, the executing entity generates a pending storyboard video corresponding to each of the multiple candidate complete director's scripts for that scene, using a multi-layered cascaded video generation mechanism. Specifically, for each target scene, multiple candidate complete director's scripts are created using a variant algorithm (e.g., A / B testing-based text generation). Each candidate complete director's script is then input into the multi-layered cascaded mechanism, where the script text is first parsed into structured visual data, then a dynamic model is applied to add motion trajectories and synchronize temporal data, and finally, the rendering engine is invoked to integrate audio and video elements.

[0062] The evaluation and cropping process can be carried out through a two-stage screening: in the first round, versions with scores below the threshold (e.g., <0.7) are quickly eliminated through a pre-trained quality prediction model; the remaining candidate videos enter the human simulation evaluation stage, and the final score is calculated through an audience reaction prediction algorithm.

[0063] This embodiment discloses a method for generating and filtering multiple alternative complete director's scripts for specific scenes. For target scenes corresponding to key plot points across multiple scenes, it utilizes the actual director's script design for each target scene and multiple matching director's script content templates. A tree search algorithm is used to generate multiple alternative complete director's scripts corresponding to each target scene in parallel, thereby automating the generation of alternative complete director's scripts, reducing repetitive manual labor, shortening the pre-production cycle, and enhancing the feasibility of creative ideas. For target scenes corresponding to key plot points across multiple scenes, the multiple alternative complete director's scripts for each target scene are processed according to a multi-layered cascaded video generation mechanism to generate pending storyboard videos corresponding to each alternative complete director's script. The target storyboard video is determined based on the actual effect evaluation of each pending storyboard video, and all alternative complete director's scripts that do not correspond to the target storyboard video are trimmed. This ensures creative diversity and improves the quality of the final film through precise filtering, achieving intelligent production and optimization of target storyboard videos.

[0064] Furthermore, the executing entity can also provide feedback to the evaluation object on the complete director's script generated for the target scene corresponding to the key plot in multiple scenes, and adjust the actual director's script design and / or actual director's script content used to generate the complete director's script based on the feedback received from the evaluation object.

[0065] The evaluation subjects can submit feedback through a standardized form, which includes three types of actionable items: first, shot-level parameter adjustments (e.g., "extend close-up duration from 3 seconds to 5 seconds"), second, narrative logic corrections (e.g., "remove redundant actions of the protagonist in scene A"), and third, style preference annotations (e.g., "increase the intensity of the cool-toned filter"). The implementing entity can use a natural language processing model to transform the textual feedback into structured instructions: for vague descriptions, it automatically associates quantifiable parameters such as shot duration and editing frequency to generate specific adjustment suggestions.

[0066] Based on any of the above embodiments, the executing entity may specifically utilize a preset scene analysis agent to determine the story content and corresponding plot shooting type of each scene in the episode script; utilize a preset director script design agent to determine a matching director script design template based on the story content of each scene, and generate an actual director script design based on the matching director script design template; utilize a preset director script content generation agent to determine a matching director script content template based on the story content of each scene, and generate a complete director script containing the actual director script content corresponding to each scene based on the actual director script design of each scene and the matching director script content template.

[0067] In this embodiment, different intelligent agents are used collaboratively to complete the complete director's script, which improves the reliability of the results of a single stage and strengthens the connection between different intelligent agents, thus helping to better control the overall quality of the generated complete director's script.

[0068] Based on the above embodiments, in order to maximize the similarity between the generated actual director's script design and / or the actual director's script content and the expected content, please also refer to... Figure 4 , Figure 4 A flowchart of a method for guiding modifications based on correctness evaluation provided in this disclosure embodiment is included in process 400, which includes the following steps:

[0069] Step 401: Evaluate the correctness of the actual director's script design and / or actual director's script content generated for each scene, and generate the first modification suggestion if the correctness evaluation fails.

[0070] This step aims to have the aforementioned implementing entity evaluate the correctness of the actual director's script design and / or actual director's script content generated for each scene, and generate a first modification suggestion if the correctness evaluation fails. Specifically, the correctness evaluation assesses whether the actual similarity between the generated actual director's script design and the matching director's script design template used is lower than a preset similarity threshold, and / or whether the actual similarity between the actual director's script content and the matching director's script content template used is lower than a preset similarity threshold.

[0071] Step 402: Adjust the currently generated actual director's script design and / or actual director's script content according to the first modification suggestion until the modified actual director's script design and / or actual director's script content pass the correctness evaluation.

[0072] Building upon step 401, this step aims to have the aforementioned executing entity adjust the currently generated actual director's script design and / or actual director's script content according to the first modification feedback until the modified actual director's script design and / or actual director's script content passes the correctness evaluation. The adjustment operations include: adjusting the actual director's script design and / or actual director's script content in conjunction with the first modification feedback without changing the corresponding template used; and regenerating a new actual director's script design and / or new actual director's script content using a new corresponding template in conjunction with the first modification feedback.

[0073] Quality control of the director's script can be achieved through the closed-loop evaluation and dynamic correction mechanism described above, thereby ensuring that the generated content meets professional film and television production standards. This mechanism triggers quality alarms by calculating the similarity between the generated script and the matching template, and performs a structured evaluation of the actual director's script design and content. When the correctness evaluation fails, adjustments are made to the currently generated actual director's script design and / or actual director's script content. Specifically, for local deviations, the original template is retained and modification suggestions are injected, and automatic optimization is achieved through a parameter fine-tuning engine; for global inconsistencies (such as emotional tone conflicts), a template replacement process is initiated, that is, a more suitable candidate template is retrieved from the template library based on the scene tags and the current defect type, and the script framework is regenerated. A version comparison tool is introduced during the correction process to visually display the differences before and after the modification, assisting in manual confirmation.

[0074] The correctness evaluation-guided modification method disclosed in this embodiment first evaluates the correctness of the actual director's script design and / or actual director's script content generated for each scene. If the correctness evaluation fails, a first modification suggestion is generated. Then, the currently generated actual director's script design and / or actual director's script content are adjusted according to the first modification suggestion until the modified actual director's script design and / or actual director's script content passes the correctness evaluation. This realizes automated quality inspection and real-time iterative optimization of the actual director's script, transforming the traditional experience-based manual correction into a rule-driven standardized process.

[0075] Building upon the aforementioned understanding of using multiple different agents to generate a complete director's script, in this embodiment, the executing entity can also utilize a preset human-like script evaluation agent to evaluate the correctness of the actual director's script design generated by the director's script design agent and / or the actual director's script content generated by the director's script content generation agent. If the correctness evaluation fails, a first modification suggestion is generated and returned to the director's script design agent and / or the director's script content generation agent. This allows the director's script design agent and / or the director's script content generation agent to adjust the currently generated actual director's script design and / or actual director's script content based on the first modification suggestion, until the modified actual director's script design and / or actual director's script content pass the correctness evaluation of the human-like script evaluation agent.

[0076] Based on the above embodiments, to deepen the understanding of the specific process of generating episode videos from a complete director's script, please also refer to... Figure 5 , Figure 5 A flowchart of a multi-layer cascaded video generation method provided in this disclosure embodiment is included in process 500, comprising the following steps:

[0077] Step 501: Generate the keyframes for each scene based on the script content for each shot in the complete director's script for each scene.

[0078] The aforementioned execution entity can generate key image frames for each scene based on the script content for each scene in the complete director's script for each scene.

[0079] Specifically, the executing entity can use text-based and graph-based graph agents to generate key image frames for each scene from the script content of each shot in the complete director's script for each scene. For example, the input text instructions are first parsed into composition parameters, and the basic image is output. Then, the graph-based graph agent performs cross-shot consistency checks on the basic image.

[0080] Step 502: Based on the key image frames of each storyboard, generate the corresponding storyboard video carrying action behavior information and interactive dialogue audio.

[0081] Based on step 501, the aforementioned execution entity can generate a storyboard video carrying action behavior information and interactive dialogue audio for each storyboard based on the key image frames of each storyboard. Specifically, the execution entity can use an action interpolation engine to calculate pixel displacement paths based on optical flow, compress redundant frames through a time warp algorithm to ensure that the duration of a single storyboard precisely matches the duration set in the script, and drive three-dimensional lip-sync through a phoneme decomposition model for the interactive dialogue audio to ensure that the lip shape changes every second are strictly aligned with the audio waveform.

[0082] Step 503: According to the time information determined for each shot in the complete director's script for each scene, stitch together the shot videos of each shot in a timeline manner to obtain the scene video corresponding to each scene.

[0083] Based on step 502, the aforementioned execution entity can stitch together the storyboard videos of each scene in a timeline manner according to the time information determined for each scene in the complete director's script for each scene, thereby obtaining the scene video corresponding to each scene. When stitching together the storyboard videos of each scene, a preset transition strategy can be adopted: when the editing rhythm of adjacent scenes exceeds a preset threshold, a fade-in / fade-out effect is automatically inserted to avoid visual jumps; for non-linear narratives, an independent timeline is created and color parameters are marked, and the corresponding filter is activated during stitching.

[0084] Step 504: Based on the time information of each scene, stitch together the scene videos of each scene in a timeline manner to obtain the episode short videos corresponding to the episode script.

[0085] Based on step 503, the aforementioned execution entity can splice the segment videos of each segment in a timeline manner according to the time information of each segment to obtain the episode short video corresponding to the episode script.

[0086] The multi-layered cascaded video generation method disclosed in this embodiment first generates key image frames for each scene based on the script content of each shot in the complete director's script for each scene. Then, based on the key image frames of each shot, a shot video carrying action behavior information and interactive dialogue audio for the corresponding shot is generated. Next, according to the time information determined for each shot in the complete director's script for each scene, the shot videos of each shot are spliced ​​together in a timeline manner to obtain the scene video corresponding to each scene. Finally, according to the time information of each scene, the scene videos of each scene are spliced ​​together in a timeline manner to obtain the episode short video corresponding to the episode script. This embodiment, by processing script content and synthesizing audiovisual elements, finally outputs video results that meet the requirements of the episode script, realizing the fully automated generation process from a complete director's script to episode short videos, reducing manual labor and improving the efficiency of video production.

[0087] Furthermore, to maximize the quality of the generated video, the implementing entity can also conduct a video quality evaluation for each storyboard video and / or each segment video, and generate a second modification suggestion if the video quality evaluation fails. The video quality evaluated includes at least one quality indicator among: smoothness of image, consistency of shot transitions, naturalness of movement, and audio-visual synchronization. The currently generated storyboard video and / or segment video is adjusted according to the second modification suggestion until the regenerated storyboard video and / or segment video content passes the video quality evaluation.

[0088] Similar to the example above, the aforementioned execution entity can also specifically evaluate the video quality of each storyboard video and / or each segment video through a preset human-like video quality evaluation intelligent agent, and generate a second modification suggestion when the video quality evaluation fails. The currently generated storyboard video and / or segment video is then adjusted according to the second modification suggestion until the content of the regenerated storyboard video and / or segment video passes the video quality evaluation of the human-like video quality evaluation intelligent agent.

[0089] In this embodiment, the video quality evaluation and dynamic correction mechanism achieves quality control of generated content through multi-dimensional quantitative analysis and closed-loop feedback optimization. This technical solution first establishes a multi-level evaluation system for video quality: image smoothness is detected through inter-frame optical flow rate change; shot continuity is verified using a feature point matching algorithm; motion naturalness is physically validated based on a biomechanical model; and audio-visual synchronization is precisely measured by comparing the millisecond-level timestamps of audio waveforms and lip movements. When the video quality evaluation fails, the executing entity can initiate the following correction strategies: for image smoothness issues, a frame interpolation algorithm is used to generate transition frames; for shot continuity issues, a scene re-rendering module is triggered to eliminate visual jumps by adjusting lighting parameters and spatial coordinates; for motion naturalness defects, a physics simulator is called to recalculate the motion trajectory; and for audio-visual synchronization issues, the audio track timeline is dynamically adjusted or lip-sync animation data is regenerated. After each correction, an incremental evaluation is automatically performed, only partially regenerating substandard segments to avoid wasting resources on full reconstruction.

[0090] To enhance understanding, this disclosure also includes... Figure 6 This paper presents a director's script and short drama video generation system based on multi-agent collaboration. The system is composed of text-based agents, image-based agents, and video-based agents in its overall architecture. Through the collaborative mechanism of heterogeneous multi-agents, it realizes the intelligentization of the entire process of director's script generation, tree search optimization, cascaded video generation, and interactive feedback.

[0091] The core innovations of this system include: 1) a template-based director's script and video generation mechanism, which uses a library of preset templates for different types of plots to achieve structured and standardized script design and video generation instruction template matching; 2) a multi-level cascaded video generation mechanism, which achieves a high-quality and controllable generation process from keyframes to shot videos and then to full-length splicing; 3) a tree-search-based director's script optimization and feedback pruning mechanism, which links script generation with video result quality optimization; 4) an interactive creation mechanism, which collects user feedback at key nodes to achieve dynamic creative adjustments; and 5) a closed-loop optimization mechanism based on video quality evaluation, which uses a video evaluation agent to achieve multi-dimensional quality feedback and adaptive model adjustment.

[0092] The following sections provide key explanations of the director's script generation stage and the video generation stage:

[0093] I. Director's Script Generation Stage

[0094] First, the system uses a large language model to analyze each act (an independent plot point in a scene occurring within a continuous timeframe) in the original script, obtaining a story analysis and recommended shooting type for each act. The analysis results returned in this stage will guide subsequent script design and specific script generation processes such as storyboarding.

[0095] Then, after obtaining the story analysis and plot shooting type analysis results for each act of the script, the system will call the director script design agent to design the script. Based on the analysis results obtained in the preliminary stage, the system will generate a formatted director design script for different plot types, including the act structure, the shot division corresponding to the act, the temporal relationship of the shots, the overall shooting techniques, etc., and at the same time, link each shot to a story in the script.

[0096] Next, the system calls another director script generation agent. Based on the script design, it further generates script information such as shooting techniques, character dialogue, action logic, and video length for each storyboard video, and finally obtains the complete director script after script conversion.

[0097] Finally, when the director script design agent generates the director script design and the director script generation agent generates the director script content, the system will also call another script evaluation agent based on a language model to evaluate the script and ensure that the generated script content meets the evaluation criteria.

[0098] In this implementation, a template-based reference generation mechanism is designed during the generation of these agents. Specifically, the system predefines numerous director's script design templates, director's script content templates, and video action templates corresponding to different types of plots. These templates are categorized according to plot type. When the director's script design agent generates the script, it selects and retrieves relevant director's script design templates for reference. Furthermore, when the script evaluation agent evaluates the output of the director's script design agent, it also searches to see if the newly generated script is similar to the reference template. A similar approach is used when the director's script generation agent generates the script.

[0099] In this implementation, to ensure the rationality of the script design, this invention employs a tree search algorithm to generate multiple alternative director script schemes (branch nodes) in parallel for key plot segments and shot nodes. During the subsequent video generation stage, branches that do not meet expectations are pruned based on feedback from the generated effects (such as image smoothness and motion coordination), retaining only the optimal script structure. This mechanism ensures the diversity of script generation and consistency with the final result orientation, enabling the director script and video effects to form a dynamic co-optimization relationship.

[0100] In this implementation, at key points (such as scene transitions, emotional climaxes, and action transitions), the system proactively interacts with the user, displaying a draft script and sample shots, and dynamically adjusting the script content and parameter settings based on user feedback. This human-machine collaboration mechanism ensures that the generated content not only conforms to the user's creative vision but also maintains automation and efficiency.

[0101] II. Video Generation Stage

[0102] In this stage, the system employs a three-level cascaded structure to generate video: Level 1 (Keyframe Generation Layer): Text-to-Graphic and Image-to-Graphic agents generate keyframes for each storyboard shot; Level 2 (Storyboard Video Generation Layer): Based on the keyframe input, the storyboard video generation agent generates short video clips with actions and dialogue; Level 3 (Video Compositing Layer): The system automatically stitches together the storyboard videos according to the timeline defined in the director's script, matching dialogue, ambient sounds, and background music to generate a complete draft of the short drama video. This hierarchical generation method significantly improves the controllability and rendering efficiency of the generation process while ensuring visual continuity.

[0103] In this implementation, the system integrates text-based, image-based, and video-based agents at this stage, achieving task-level parallelism and information sharing through a unified scheduling mechanism. The director script output directly drives image and video generation, while the video generation results are fed back to the director script optimization module, enabling cross-modal collaboration. This hybrid collaboration mechanism significantly improves generation efficiency and multimodal consistency.

[0104] In this implementation, the system is equipped with a video evaluation agent to perform multi-dimensional quality assessments of the generated video, including indicators such as video smoothness, shot transition consistency, motion naturalness, and audio-visual synchronization. The evaluation results are fed back in real time to the director's script generation and video generation agents, guiding the next round of iterative optimization and achieving closed-loop adaptive improvement of the generated content.

[0105] The technical solution disclosed in this implementation firstly matches the determined plot shooting type with director's script design templates and director's script content templates derived from highly-rated historical films and television works. This transforms professional directing experience into reusable parameters, ensuring that the camera language conforms to industry standards. Secondly, the templates are used to generate actual director's script designs and fill in script content, addressing the issues of fragmented camera language and uncontrolled pacing in the generated videos. Finally, a multi-layered cascaded video generation mechanism enables layer-by-layer quality control, ensuring character consistency, action coherence, and audio-visual synchronization. This solution achieves efficient production of short drama videos through the use of structured templates and a layered generation mechanism.

[0106] Further reference Figure 7 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a short drama video generation device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0107] like Figure 7As shown, the short drama video generation device 700 of this embodiment may include: an episode script analysis unit 701, a template matching unit 702, a complete director's script generation unit 703, and a multi-layer cascaded video generation unit 704. The system includes: Episode Script Analysis Unit 701, configured to determine the story content and corresponding shooting type of each scene in the episode script; Template Matching Unit 702, configured to determine the director's script design template and director's script content template that match the shooting type of the scene, with pre-recorded correspondences between different shooting types and different director's script design templates and different director's script content templates, which are obtained by templated processing of director's scripts of historical film and television works with actual scores exceeding the preset score; Complete Director's Script Generation Unit 703, configured to generate an actual director's script design using the story content of each scene and the matched director's script design template, and to generate a complete director's script containing the actual director's script content corresponding to each scene using the actual director's script design and the matched director's script content template; and Multi-level Cascading Video Generation Unit 704, configured to generate episode short videos corresponding to the episode script based on the complete director's script of each scene and the preset multi-level cascading video generation mechanism.

[0108] In this embodiment, the specific processing and technical effects of the episode script analysis unit 701, template matching unit 702, complete director's script generation unit 703, and multi-layer cascaded video generation unit 704 in the short drama video generation device 700 can be referred to respectively. Figure 2 The relevant descriptions of steps 201-204 in the corresponding embodiments will not be repeated here.

[0109] In some optional implementations of this embodiment, the episode script analysis unit 701 is further configured to: determine the actual plot of the corresponding character in the corresponding scene of the corresponding scene based on the script content of each episode script; determine the shooting type to be used to shoot the video content of the actual plot of the corresponding character in the corresponding scene for each episode, and obtain the plot shooting type corresponding to each episode.

[0110] In some optional implementations of this embodiment, the director's script design template includes at least one of the following template configuration items to be determined: scene structure, scene segmentation corresponding to the scene, temporal relationship between each scene, and shooting technique used.

[0111] In some optional implementations of this embodiment, the complete director's script generation unit 703 is further configured to: generate the actual script content of each scene by using a director's script content template that matches the corresponding scene, thereby obtaining a complete director's script corresponding to each scene; wherein, the actual script content includes at least one of the following: shooting techniques to be used in different scenes, character dialogue, action logic, and video duration.

[0112] In some optional implementations of this embodiment, the complete director script generation unit 703 includes: a parallel generation of multiple alternative script subunits, which are configured to generate multiple alternative complete director scripts corresponding to each target scene in parallel using a tree search algorithm, based on the actual director script design of each target scene and multiple matching director script content templates for the target scenes corresponding to key plot points in multiple scenes.

[0113] In some optional implementations of this embodiment, the multi-layer cascaded video generation unit 704 includes: a storyboard video actual effect trimming subunit, which is configured to, for target scenes corresponding to key plot points in multiple scenes, generate undetermined storyboard videos corresponding to each candidate complete director's script for each target scene according to the multi-layer cascaded video generation mechanism, determine the target storyboard video based on the actual effect evaluation of each undetermined storyboard video, and trim all candidate complete director's scripts that do not correspond to the target storyboard video.

[0114] In some optional implementations of this embodiment, the short drama video generation device 700 may further include: an object feedback adjustment unit, configured to feed back the complete director's script generated for the target scene corresponding to the key plot in multiple scenes to the evaluation object, and adjust the actual director's script design and / or actual director's script content used to generate the complete director's script based on the feedback received from the evaluation object.

[0115] In some optional implementations of this embodiment, the short drama video generation device 700 may further include: a correctness evaluation unit, configured to evaluate the correctness of the actual director script design and / or actual director script content generated for each scene, and generate a first modification suggestion when the correctness evaluation fails; wherein, the correctness evaluation is used to evaluate whether the actual similarity between the generated actual director script design and the matching director script design template used is lower than a preset similarity threshold and / or whether the actual similarity between the actual director script content and the matching director script content template used is lower than a preset similarity threshold; a first modification unit, configured to adjust the currently generated actual director script design and / or actual director script content according to the first modification suggestion until the modified actual director script design and / or actual director script content passes the correctness evaluation; wherein, the adjustment operation includes: adjusting the actual director script design and / or actual director script content in conjunction with the first modification suggestion without changing the corresponding template used, and regenerating a new actual director script design and / or new actual director script content using a new corresponding template in conjunction with the first modification suggestion.

[0116] In some optional implementations of this embodiment, the episode script analysis unit 701 is further configured to: use a preset scene analysis agent to determine the story content and corresponding plot shooting type of each scene contained in the episode script;

[0117] The template matching unit 702 and the complete director's script generation unit 703 are further configured to: use a preset director's script design agent to determine a matching director's script design template based on the story content of each act, and generate an actual director's script design based on the matching director's script design template; use a preset director's script content generation agent to determine a matching director's script content template based on the story content of each act, and generate a complete director's script containing the actual director's script content corresponding to each act based on the actual director's script design of each act and the matching director's script content template.

[0118] In some optional implementations of this embodiment, the correctness evaluation unit and the first modification unit are further configured to: use a preset human-like script evaluation agent to evaluate the correctness of the actual director script design generated by the director script design agent and / or the actual director script content generated by the director script content generation agent; generate a first modification opinion if the correctness evaluation fails; and return the first modification opinion to the director script design agent and / or the director script content generation agent, so that the director script design agent... and / or the intelligent agent that generates the director's script contentBased on the first modification suggestion, the currently generated actual director script design and / or actual director script content are adjusted until the modified actual director script design and / or actual director script content pass the correctness evaluation of the humanoid script evaluation agent.

[0119] In some optional implementations of this embodiment, the multi-layer cascaded video generation unit 704 includes: a key image frame generation subunit, configured to generate key image frames for each scene based on the script content for each scene in the complete director's script for each scene; a scene video generation subunit, configured to generate scene video carrying action behavior information and interactive dialogue audio for each scene based on the key image frames for each scene; a scene video splicing subunit, configured to splice the scene videos of each scene in a timeline manner according to the time information determined for each scene in the complete director's script for each scene, to obtain a scene video corresponding to each scene; and a scene video splicing subunit, configured to splice the scene videos of each scene in a timeline manner according to the time information of each scene, to obtain a short episode video corresponding to the episode script.

[0120] In some optional implementations of this embodiment, the key image frame generation subunit is further configured to: generate key image frames for the corresponding scenes from the script content of each scene in the complete director's script for each scene using text-to-image and image-to-image agents.

[0121] In some optional implementations of this embodiment, the short drama video generation device 700 may further include: a video quality evaluation unit, configured to evaluate the video quality of each storyboard video and / or each segment video, and generate a second modification suggestion when the video quality evaluation fails; wherein the video quality evaluated includes at least one quality indicator among: picture smoothness, shot transition consistency, action naturalness, and audio-visual synchronization; and a second modification unit, configured to adjust the currently generated storyboard video and / or segment video according to the second modification suggestion until the content of the regenerated storyboard video and / or segment video passes the video quality evaluation.

[0122] In some optional implementations of this embodiment, the video quality evaluation unit and the second modification unit are further configured to: perform video quality evaluation on each storyboard video and / or each segment video through a preset human-like video quality evaluation intelligent agent, generate a second modification opinion when the video quality evaluation fails, and adjust the currently generated storyboard video and / or segment video according to the second modification opinion until the content of the regenerated storyboard video and / or segment video passes the video quality evaluation of the human-like video quality evaluation intelligent agent.

[0123] This embodiment exists as a device embodiment corresponding to the above method embodiment. The short drama video generation device provided in this embodiment first matches the determined plot shooting type with director's script design templates and director's script content templates derived from high-scoring historical film and television works, thereby converting professional director experience into reusable parameters and ensuring that the camera language conforms to industry standards. Secondly, it uses the templates to generate actual director's script designs and fills in the script content to solve the problems of fragmented camera language and out-of-control rhythm in the generated video. Finally, the multi-layer cascaded video generation mechanism can achieve layer-by-layer quality control to ensure character consistency, action coherence, and audio-visual synchronization. This solution achieves efficient production of short drama videos by using structured templates and a layered generation mechanism.

[0124] According to embodiments of this disclosure, this disclosure also provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to implement the short drama video generation method described in any of the above embodiments.

[0125] According to embodiments of this disclosure, this disclosure also provides a readable storage medium storing computer instructions that, when executed by a computer, enable the short drama video generation method described in any of the above embodiments.

[0126] According to embodiments of this disclosure, this disclosure also provides a computer program product that, when executed by a processor, can implement the steps of the short drama video generation method described in any of the above embodiments.

[0127] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0128] like Figure 8As shown, device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 802 or a computer program loaded from storage unit 808 into random access memory (RAM) 803. RAM 803 may also store various programs and data required for the operation of device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via bus 804. Input / output (I / O) interface 805 is also connected to bus 804.

[0129] Multiple components in device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of monitors, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0130] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as the short video generation method. For example, in some embodiments, the short video generation method may be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the short video generation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the short video generation method by any other suitable means (e.g., by means of firmware).

[0131] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0132] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0133] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0134] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0135] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0136] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0137] The technical solution of this disclosure firstly matches the determined plot shooting type with director's script design templates and director's script content templates derived from highly-rated historical films and television works, thereby transforming professional directing experience into reusable parameters and ensuring that the camera language conforms to industry standards. Secondly, it uses templates to generate actual director's script designs and fills in script content to solve the problems of fragmented camera language and uncontrolled pacing in the generated videos. Finally, a multi-layered cascaded video generation mechanism enables layer-by-layer quality control, ensuring character consistency, action coherence, and audio-visual synchronization. This solution achieves efficient production of short drama videos through the use of structured templates and a layered generation mechanism.

[0138] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0139] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for generating short drama videos, comprising: Determine the story content of each scene in the episode script and the corresponding shooting type; Determine the director's script design template and director's script content template that match the shooting type of the plot. The correspondence between different shooting types of plots and different director's script design templates and different director's script content templates is recorded in advance. The director's script design template and director's script content template are obtained by templated processing of the director's scripts of historical film and television works with actual scores that exceed the preset score. The actual director's script design is generated by using the story content of each act and the matching director's script design template. Then, the complete director's script containing the actual director's script content is generated for each act by using the actual director's script design and the matching director's script content template. Based on the complete director's script for each scene and the preset multi-layered cascaded video generation mechanism, short videos corresponding to the episode scripts are generated. The multi-layered cascaded video generation mechanism generates static visual anchor points based on the storyboard descriptions in the director's script, uses a text-to-image intelligent agent to parse text instructions into spatial composition parameters and output basic keyframes, uses an image-to-image intelligent agent to perform cross-storyboard consistency optimization, identifies core elements through a feature extraction model and compares them with a global parameter library, and triggers a feature mapping correction mechanism that replaces abnormal elements while preserving the composition when a conflict is detected.

2. The method of claim 1, wherein, The determination of the story content and corresponding shooting type of each scene in the episode script includes: Based on the script content of each act contained in the episode script, determine the actual plot that occurs to the corresponding characters in the corresponding scene of the corresponding act; Determine the shooting type to be used for each scene to obtain the actual plot of the corresponding characters in the corresponding scene, and obtain the plot shooting type corresponding to each scene.

3. The method of claim 1, wherein, The director's script design template includes at least one of the following template configuration items to be determined: scene structure, scene segmentation, temporal relationship between scenes, and shooting techniques.

4. The method according to claim 1, wherein, The process involves using the actual director's script design and matching director's script content template for each scene to generate a complete director's script corresponding to each scene, including: The actual director's script design for each scene is generated by using a director's script content template that matches the corresponding scene to obtain a complete director's script for each scene; wherein, the actual script content includes at least one of the following: shooting techniques to be used in different scenes, character dialogue, action logic, and video length.

5. The method according to claim 1, wherein, The process involves using the actual director's script design and matching director's script content template for each scene to generate a complete director's script corresponding to each scene, including: For target scenes with key plot points in multiple scenes, multiple matching director script content templates are designed using the actual director script of each target scene, and a tree search algorithm is used to generate multiple alternative complete director scripts corresponding to each target scene in parallel. Correspondingly, the generation of episode short videos corresponding to the episode scripts, based on the complete director's script for each scene and the preset multi-layered cascading video generation mechanism, includes: For a target scene corresponding to a key plot point in multiple scenes, multiple candidate complete director scripts for each target scene are generated into a pending storyboard video corresponding to each candidate complete director script according to the multi-layer cascaded video generation mechanism. The target storyboard video is determined based on the actual effect evaluation of each pending storyboard video, and all candidate complete director scripts that do not correspond to the target storyboard video are cut off.

6. The method according to claim 1, further comprising: For a target scene corresponding to a key plot point in multiple scenes, the complete director's script generated for the target scene will be fed back to the evaluation object, and the actual director's script design and / or actual director's script content used to generate the complete director's script will be adjusted according to the feedback received from the evaluation object.

7. The method according to any one of claims 1-6, further comprising: A correctness evaluation is performed on the actual director script design and / or actual director script content generated for each scene, and a first modification suggestion is generated when the correctness evaluation fails; wherein, the correctness evaluation is used to evaluate whether the actual similarity between the generated actual director script design and the matching director script design template used is lower than a preset similarity threshold and / or whether the actual similarity between the actual director script content and the matching director script content template used is lower than a preset similarity threshold. The currently generated actual director's script design and / or actual director's script content are adjusted according to the first modification suggestion until the modified actual director's script design and / or actual director's script content passes the correctness evaluation; wherein, the adjustment operation includes: adjusting the actual director's script design and / or actual director's script content in conjunction with the first modification suggestion without changing the corresponding template used, and regenerating a new actual director's script design and / or new actual director's script content using a new corresponding template in conjunction with the first modification suggestion.

8. The method according to claim 7, wherein, The determination of the story content and corresponding shooting type of each scene in the episode script includes: The pre-set scene analysis agent is used to determine the story content and corresponding plot shooting type of each scene in the episode script; The process involves determining a director's script design template and a director's script content template that match the plot shooting type, generating an actual director's script design using the story content of each scene and the matching director's script design template, and then generating a complete director's script containing the actual director's script content corresponding to each scene using the actual director's script design and the matching director's script content template, including: The pre-set director script design agent determines the matching director script design template based on the story content of each scene, and generates the actual director script design based on the matching director script design template. Using a pre-set director's script content, an intelligent agent determines a matching director's script content template based on the story content of each act, and generates a complete director's script containing the actual director's script content for each act based on the actual director's script design and the matching director's script content template.

9. The method according to claim 8, wherein, The process of evaluating the correctness of the actual director's script design and / or actual director's script content generated for each scene, and generating a first modification suggestion when the correctness evaluation fails, and adjusting the currently generated actual director's script design and / or actual director's script content according to the first modification suggestion until the modified actual director's script design and / or actual director's script content passes the correctness evaluation, includes: A preset human-like script evaluation agent is used to evaluate the correctness of the actual director script design generated by the director script design agent and / or the actual director script content generated by the director script content generation agent. If the correctness evaluation fails, a first modification suggestion is generated and returned to the director script design agent and / or the director script content generation agent. This allows the director script design agent and / or the director script content generation agent to adjust the currently generated actual director script design and / or actual director script content according to the first modification suggestion, until the modified actual director script design and / or actual director script content pass the correctness evaluation of the human-like script evaluation agent.

10. The method according to claim 1, wherein, The method, based on the complete director's script for each scene and a pre-set multi-layered cascading video generation mechanism, generates episode-specific short videos corresponding to the episode scripts, including: Based on the script content for each shot in the complete director's script for each scene, generate the keyframes for the corresponding shot. Based on the key image frames of each storyboard, generate a storyboard video that carries action and behavior information and interactive dialogue audio for the corresponding storyboard. Based on the time information determined for each shot in the complete director's script for each scene, the shot videos of each shot are spliced ​​together in a timeline manner to obtain the scene video corresponding to each scene. Based on the time information of each scene, the scene videos of each scene are spliced ​​together in a timeline manner to obtain the episode short videos corresponding to the episode script.

11. The method according to claim 10, wherein, The step of generating keyframes for each scene based on the script content for each shot in the complete director's script for each scene includes: The script content for each shot in the complete director's script for each scene is used to generate key image frames for the corresponding shot using text-to-image and image-to-image intelligent agents.

12. The method according to claim 10 or 11, further comprising: A video quality evaluation is performed on each storyboard video and / or each scene video, and a second modification suggestion is generated if the video quality evaluation fails; wherein, the video quality evaluated includes at least one of the following quality indicators: smoothness of the picture, consistency of shot transitions, naturalness of movement, and audio-visual synchronization; Adjust the currently generated storyboard video and / or segmented video according to the second modification suggestion until the content of the regenerated storyboard video and / or segmented video passes the video quality evaluation.

13. The method according to claim 12, wherein, The process of evaluating the video quality of each storyboard video and / or each segment video, and generating a second modification suggestion when the video quality evaluation fails, and adjusting the currently generated storyboard video and / or segment video according to the second modification suggestion until the content of the regenerated storyboard video and / or segment video passes the video quality evaluation, includes: The human-like video quality evaluation agent performs video quality evaluation on each storyboard video and / or each segment video. If the video quality evaluation fails, a second modification suggestion is generated. The currently generated storyboard video and / or segment video are adjusted according to the second modification suggestion until the content of the regenerated storyboard video and / or segment video passes the video quality evaluation of the human-like video quality evaluation agent.

14. A short drama video generation device, comprising: The episode script analysis unit is configured to determine the story content of each scene contained in the episode script and the corresponding plot shooting type; The template matching unit is configured to determine the director script design template and director script content template that match the plot shooting type. The correspondence between different plot shooting types and different director script design templates and different director script content templates is pre-recorded. The director script design template and director script content template are obtained by template processing of the director scripts of historical film and television works with actual scores exceeding the preset score. The complete director's script generation unit is configured to generate an actual director's script design using the story content of each act and the matching director's script design template, and to generate a complete director's script containing the actual director's script content corresponding to each act using the actual director's script design of each act and the matching director's script content template. The multi-layer cascaded video generation unit is configured to generate episode short videos corresponding to the episode scripts based on the complete director's script for each episode and a preset multi-layer cascaded video generation mechanism. The multi-layer cascaded video generation mechanism generates static visual anchor points based on the storyboard descriptions in the director's script, uses a text-to-image intelligent agent to parse text instructions into spatial composition parameters to output basic keyframes, uses an image-to-image intelligent agent to perform cross-storyboard consistency optimization, identifies core elements through a feature extraction model and compares them with a global parameter library, and triggers a feature mapping correction mechanism that replaces abnormal elements while retaining the composition when a conflict is detected.

15. An electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the short drama video generation method according to any one of claims 1-13.

16. A non-transitory computer-readable storage medium storing computer instructions for causing the computer to perform the short video generation method according to any one of claims 1-13.

17. A computer program product comprising a computer program that, when executed by a processor, implements the steps of the short drama video generation method according to any one of claims 1-13.

Images