An intelligent director scheduling method and system for AI boutique video content production

Abstract

The application discloses an intelligent director scheduling method and system for AI fine film and television content production, which comprises the following steps: obtaining a script text and performing semantic analysis, extracting scene elements and emotion curves; generating a structured prompt word sequence based on the scene elements and emotion curves, arranging the shot switching sequence and transition logic, and outputting a shot sequence with coherent narration; calling an AIGC video generation model according to the shot sequence to generate a video, and taking the tail frame information of the current shot as the initial latent space constraint condition for the next shot generation in the generation process, so as to maintain the motion coherence and picture space logic consistency when switching shots. The application also provides role consistency control, scene style unification, and adaptive parameter optimization and rollback mechanism based on quality evaluation. The application realizes end-to-end automatic production from a script to a finished film, and significantly improves the narrative coherence, role consistency and production efficiency of AI film and television content generation.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence content generation technology, and in particular relates to an intelligent director scheduling method and system for producing high-quality AI-generated film and television content. Background Technology

[0002] In recent years, with the rapid development of diffusion models and large-scale video generation models, AIGC technology has shown great potential in the field of film and television content production. However, existing AIGC film and television production tools, such as Runway Gen-3, Pika 2.0, and Keling AI, mainly focus on the generation of single clips or short videos, and cannot yet support the automated production process from a complete script to a multi-shot narrative film. The main technical shortcomings are as follows:

[0003] First, the conversion from script to prompts relies on manual intervention. In traditional AIGC video generation processes, users must manually convert the script text into English prompts that the video generation model can understand. This lacks structured analysis of the script's semantics and the ability to automatically generate prompts, resulting in inconsistent prompt quality and a deviation between the generated results and the script's intent. The manual prompt writing process for a 20-episode AI short drama typically takes 40 to 60 hours.

[0004] Second, the shot sequences lack narrative structure. Existing AIGC tools generate video clips that lack progressive shot gradations, seamless motion transitions, and logical transitions, failing to form coherent narrative segments and resulting in generally low viewer completion rates. Tests show that clips generated by existing technology receive a narrative coherence score of only about 6.1 out of 10, with a viewer completion rate of less than 35%.

[0005] Third, character appearances are inconsistent across shots. AIGC video generation models generate each frame based on random latent space sampling. The same character can exhibit inconsistencies in facial features, clothing details, and body proportions across different shots, severely impacting narrative coherence and the viewing experience. Existing solutions achieve an average facial similarity of only about 0.62 across shots.

[0006] Fourth, there is significant visual style drift within the same scene. The AIGC model is highly sensitive to style descriptions in prompts; even subtle differences can result in significant variations in color tones, lighting, and texture, leading to inconsistent visual styles across multiple shots within the same scene. Blind audience ratings for style consistency are only around 5.2 points.

[0007] Fifth, the configuration of generation parameters relies heavily on experience. The setting of generation parameters such as sampling steps, guidance coefficients, and resolution depends primarily on user experience, lacking an adaptive optimization mechanism based on quality assessment and computational resource constraints. This makes it difficult to balance generation quality and efficiency. The average generation time per shot is approximately 8 minutes, with GPU resource utilization at only about 45%.

[0008] The closest existing technology, CN116789012A, discloses an artificial intelligence-based short video generation method, but it only supports the generation of single segments and does not involve script structure analysis, cross-shot character consistency control, scene style unification, and shot sequence arrangement. Furthermore, the generation parameters are fixed and cannot achieve adaptive optimization. Summary of the Invention

[0009] This invention aims to address the problems in the existing AIGC film and television production process, such as lack of script-driven approaches, poor narrative coherence, inconsistent character portrayals, style drift, and experience-based configuration of generation parameters. It provides an intelligent director scheduling method and system for producing high-quality AI-generated film and television content.

[0010] The objective of this invention is achieved through the following solution.

[0011] A method for intelligent director scheduling in AI-driven production of high-quality film and television content includes the following steps:

[0012] Obtain the script text, perform semantic analysis on the script text, and extract scene elements and emotional curves;

[0013] Based on the scene elements and emotion curves, a structured cue word sequence adapted to the AIGC video generation model is generated, and the shot switching order and transition logic are arranged according to the structured cue word sequence to output a storyboard sequence with narrative coherence.

[0014] The AIGC video generation model is invoked according to the storyboard sequence to generate video. During the generation process, the last frame information of the current shot is used as the initial latent space constraint condition for the generation of the next shot, so as to maintain the motion continuity and the consistency of the spatial logic of the picture when switching shots.

[0015] In some embodiments, the semantic parsing includes: performing word segmentation, named entity recognition, and dependency parsing on the script text to identify scene location, time setting, character information, prop requirements, and camera movement cues; classifying the script text for emotion, outputting the emotion intensity value for each scene, and fitting an emotion curve using a sliding window algorithm; identifying local extreme points of the emotion curve, and determining narrative climaxes and troughs based on the local extreme points.

[0016] In some embodiments, generating a structured cue word sequence includes: based on shot size rules, camera angle semantic mapping, motion description words, and lighting atmosphere words in the shot language knowledge base, filling a preset cue word template according to the scene elements to generate standardized cue words containing subject description, environment description, style description, quality description, and negative cue words; in the cue word template, the subject description includes character name, action state, facial expression features, clothing details, and prop holding information; the environment description includes scene location, time setting, weather lighting, background elements, and depth of field information; and the style description includes art style, reference director, color tendency, and image texture information.

[0017] In some embodiments, the arrangement of shot switching order and transition logic includes: generating a shot switching order based on shot scale progression rules and motion mode connection rules, wherein the shot scale progression rules include zoom-in sequences and zoom-out sequences, and the motion mode connection rules include the continuity relationship between different types of shot motion; automatically selecting a transition mode according to the scene switching type, wherein the transition mode includes dissolve transition, hard cut transition, flash white transition and wipe transition, and generating corresponding transition prompt words.

[0018] In some embodiments, the arrangement of shot switching order and transition logic further includes: identifying narrative climax points, transition points and calm points based on the emotional curve and script conflict structure, allocating a first shot density and a first shot duration to the climax points, and allocating a second shot density and a second shot duration to the calm points, wherein the first shot density is greater than the second shot density and the first shot duration is less than the second shot duration.

[0019] In some embodiments, the method further includes a character consistency control step: extracting the facial feature vector, body feature vector, and clothing feature vector of the target character, and establishing an independent feature vector file for each character; during the video generation stage, injecting the feature vector of the target character into the AIGC video generation model in the form of adapter weights or low-rank weights.

[0020] In some embodiments, the facial feature vector is used to extract identity feature vectors through a face analysis model, the body posture feature vector is used to extract joint coordinate sequences and body proportion parameters through a skeleton detection model, and the clothing feature vector is used to extract clothing texture and color distribution features through a visual encoder; the feature vectors are stored in a vector database in JSON format, supporting multiple version iterations and rollbacks.

[0021] In some embodiments, a scene style unification step is also included: extracting color distribution features, texture features, and composition features from keyframes of reference images or reference videos to generate style feature vectors; establishing a mapping relationship between the style feature vectors and AIGC video generation model parameters; and locking style-related parameters during the generation of multiple shots in the same scene, allowing only narrative-related parameters to change.

[0022] In some embodiments, the method further includes a parameter optimization step: evaluating the quality of the generated video frames using image quality metrics and text-image alignment metrics; performing parameter search within the sampling step, guidance coefficient, and resolution parameter space with the goal of minimizing generation time and maximizing quality score, and outputting the optimal parameter combination; and dynamically adjusting the generation strategy based on the current computing resource load.

[0023] This embodiment also provides an intelligent director scheduling system for AI-powered high-quality film and television content production, including:

[0024] The script semantic parsing engine is used to semantically parse the script text and extract scene elements and emotional curves;

[0025] The intelligent scene-prompt generation module is used to generate a structured prompt sequence adapted to the AIGC video generation model based on the scene elements and emotion curves.

[0026] A multi-shot sequence scheduler is used to arrange the shot switching order and transition logic according to the structured cue word sequence, and output a storyboard sequence with narrative coherence.

[0027] A video generation parameter optimizer is used to dynamically adjust the generation parameters based on the current computing resource load, and to call the AIGC video generation model to generate video based on the storyboard sequence;

[0028] The multi-shot sequence scheduler or the video generation parameter optimizer includes an inter-frame continuity constraint submodule. The inter-frame continuity constraint submodule is used to use the last frame information of the current shot as the initial latent space constraint condition for the generation of the next shot, so as to maintain the motion continuity and spatial logic consistency during shot switching.

[0029] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0030] 1. This invention utilizes the collaborative work of a script semantic parsing engine and an intelligent storyboard-prompt generation module to automatically transform unstructured natural language scripts into structured storyboard sequences and standardized prompts. The script semantic parsing engine employs BERT and RoBERTa pre-trained models for deep semantic mining of the script text. The intelligent storyboard-prompt generation module incorporates a complete camera language knowledge base and template engine, automatically determining shot type, camera angle, movement method, and shot duration based on scene elements, emotional curves, and narrative rhythm. This process fundamentally changes the traditional workshop-style model of "manual script reading → manual prompt writing" in AIGC production, reducing the prompt generation time per episode from 2-3 hours to 5 minutes, and increasing the alignment between prompts and script intent from 72% to 89%, making large-scale production of high-quality content possible.

[0031] 2. Existing AIGC tools generate each shot as an independent random sampling result, lacking any visual correlation between shots. This inevitably leads to "jumping" phenomena such as character teleportation, sudden scene changes, and pose jumps after stitching, which is the fundamental technical bottleneck restricting AI videos from moving from "fragmentation" to "narrative." This invention incorporates an inter-frame continuity constraint submodule into the multi-shot sequence scheduler or video generation parameter optimizer. The last frame of the current shot is mapped back to the latent space using a variational autoencoder, and its latent space feature vector is extracted as the initial latent space reference for the next shot's diffusion process. This ensures that the initial state of the next shot remains continuous with the final state of the previous shot in the latent space, guaranteeing physical continuity of the image content at the pixel level after decoding. This constraint mechanism, starting from the underlying sampling principle of the generation model, eliminates the logical break in image content during shot transitions at its root, enabling multiple AI-generated shots to seamlessly connect into continuous narrative segments, improving the narrative coherence score from 6.1 to 8.4.

[0032] 3. AIGC video generation models are based on random latent space sampling. The appearance of the same character in different shots naturally has uncontrollable randomness, which is the technical root of the "character face-changing" problem in AI videos. This invention constructs a complete character consistency control system: extracting 512-dimensional facial identity feature vectors through the InsightFace model, extracting body skeleton parameters through OpenPose, and extracting clothing texture features through ResNet-50, establishing an independent feature vector file for each character and storing it in a vector database; during video generation, the feature vectors are precisely injected into the generation model in the form of IP-Adapter or LoRA weights, so that the appearance information of the character in all shots is constrained by the same set of vectors. This solution changes from "AI random generation" to "AI generation under specified constraints", improving the cross-shot character facial similarity from 0.62 to 0.91 and the clothing consistency recognition accuracy from 45% to 88%, fundamentally solving the technical problem of character image constantly drifting during the narrative process.

[0033] 4. Visual style drift between multiple shots within the same scene is another core pain point for AI videos. This invention establishes a precise mapping relationship between style feature vectors and AIGC generation model parameters through a scene style unification module: extracting multi-dimensional style features of reference images in Lab color space, gray-level co-occurrence matrix, and edge detection layers, and establishing a one-to-one mapping between color tone, contrast level, image quality, and underlying generation parameters such as VAE decoder type, ControlNet preprocessor, and global color LUT. The generation parameter locking submodule marks the style-related parameters of all shots in the same scene as frozen by generating configuration templates, allowing only narrative-related parameters to change. This solution ensures style consistency from the underlying parameter level of the generation model rather than the cue text level, reducing color differences between shots within the same scene by 67%, and improving the audience's blind rating for style consistency from 5.2 to 8.1.

[0034] 5. In existing AIGC production, the generation parameters (sampling steps, guidance coefficients, resolution, etc.) rely entirely on manual experience for setting, making it impossible to dynamically adjust according to actual content needs and computing resource availability. This results in either insufficient image quality or wasted resources. This invention introduces a Bayesian optimization algorithm into video generation parameter search, using FID and CLIP Score as quality evaluation indicators to automatically search for the optimal combination in the parameter space. Simultaneously, based on real-time GPU memory usage and queue waiting time, it automatically switches between standard, balanced, and accelerated modes. Under high load, it seamlessly enables the LCM / SDXL-Turbo distillation model to reduce the sampling steps from 50 to 4-8, enables Tile VAE block decoding to reduce memory usage, and enables a frame interpolation model to increase the generation frame rate from 8fps to 24fps. This mechanism compresses the average generation time per shot from 8 minutes to 3.5 minutes and increases GPU resource utilization from 45% to 78%, maximizing generation throughput while ensuring a minimum image quality.

[0035] In summary, the overall value of this invention lies in transforming the traditional film and television production process—comprising a multi-role, manual decision-making chain involving a director, screenwriter, cinematographer, art director, editor, and production manager—into a data-driven, closed-loop feedback automated algorithm system. The six modules are tightly linked by a data flow logic of "analysis → planning → constraint → arrangement → generation → optimization," forming an end-to-end automated production pipeline from script input to final output. This elevates the quality rate of AIGC film and television content from less than 5% to a level that can be industrially replicated, providing a complete technical infrastructure for AI film and television to move from "segment generation" to "narrative film production." Attached Figure Description

[0036] Figure 1 This is a diagram of the overall system architecture of the present invention.

[0037] Figure 2 This is a flowchart of the script semantic parsing engine workflow of this invention.

[0038] Figure 3 This is a structural diagram of the intelligent storyboard-prompt word generation module of the present invention.

[0039] Figure 4 This is a schematic diagram illustrating the working principle of the role consistency control module of this invention.

[0040] Figure 5 This is a flowchart of the workflow for the scene style unification module of this invention.

[0041] Figure 6 This is an architecture diagram of the multi-camera sequence scheduler and video generation parameter optimizer of the present invention. Detailed Implementation

[0042] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided to make the invention more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0043] Example 1

[0044] See Figures 1-6 To address the technical challenges of lacking script-driven structured analysis and automated director decision-making, this invention provides an intelligent director scheduling method for AI-driven high-quality film and television content production. The method includes the following steps: acquiring the script text; performing semantic analysis on the script text to extract scene elements and emotional curves; generating a structured cue word sequence adapted to the AIGC video generation model based on the scene elements and emotional curves; arranging shot switching order and transition logic according to the structured cue word sequence to output a storyboard sequence with narrative coherence; and calling the AIGC video generation model to generate video based on the storyboard sequence. Through these steps, unstructured natural language scripts are automatically transformed into structured storyboard decision instructions, achieving automated translation from literary text to audiovisual language and solving the problems of low efficiency and uncontrollable quality in manual conversion.

[0045] To address the technical problem of lacking motion continuity and spatial consistency between generated segments, this invention uses the last frame information of the current shot as the initial latent space constraint for the next shot during video generation. This step uses the latent space features of the last frame of the previous shot as the initial noise condition for the diffusion process of the next shot. This ensures that the next shot inherits the image layout, object positions, and character poses from the last frame of the previous shot at the pixel level, thus guaranteeing a continuous transition of image content during switching in physical space logic and avoiding frame skipping phenomena such as instantaneous character movement or abrupt scene changes. Through this inter-frame constraint mechanism, motion continuity across shots is achieved at the physical pixel level of video frames, overcoming the deficiency that simply relying on text prompts cannot guarantee the consistency of spatial logic.

[0046] Preferably, the semantic parsing includes: performing word segmentation, named entity recognition, and dependency parsing on the script text to identify scene locations, time settings, character information, prop requirements, and camera movement cues; classifying the script text for emotion, outputting the emotion intensity value for each scene, and fitting an emotion curve using a sliding window algorithm; identifying local extreme points of the emotion curve, and determining narrative climaxes and troughs based on these local extreme points. The scene emotion values ​​are smoothed using a sliding window algorithm (window size is 3 scenes). When the emotion intensity value at the center point of the window is simultaneously greater than the left and right adjacent points, it is marked as a local maximum, i.e., a narrative climax; when it is simultaneously less than the left and right adjacent points, it is marked as a local minimum, i.e., a narrative trough. This step allows subsequent camera work to be differentiated according to the narrative rhythm, allocating high-density, short-duration shots at climaxes to enhance emotional impact, and low-density, long-duration shots at troughs to create a soothing atmosphere.

[0047] Preferably, the generation of the structured prompt word sequence includes: based on the shot size rules, camera angle semantic mapping, motion description words, and lighting atmosphere words in the lens language knowledge base, filling a preset prompt word template according to the scene elements to generate standardized prompt words containing subject description, environment description, style description, quality description, and negative prompt words. In the prompt word template, the subject description includes character name, action state, facial expression characteristics, clothing details, and prop holding information; the environment description includes scene location, time setting, weather lighting, background elements, and depth of field information; and the style description includes art style, reference director, color tendency, and image texture information. By combining structured template filling with natural language fluency processing, the completeness and standardization of the prompt words are ensured, overcoming the problems of arbitrary writing and omission of key information in manually crafted prompt words.

[0048] Preferably, the arrangement of shot switching order and transition logic includes: generating the shot switching order based on shot scale progression rules and motion mode connection rules, wherein the shot scale progression rules include a push-in sequence (long shot → wide shot → medium shot → close-up → extreme close-up, used for emotional escalation) and a pull-back sequence (extreme close-up → close-up → medium shot → wide shot → long shot, used for emotional release), and the motion mode connection rules include a push-in shot followed by a fixed shot (stabilizing emotions), a pull-out shot followed by a panning shot (expanding the field of view), a follow shot followed by a fixed shot (stopping movement), and a panning shot followed by a push-in shot (focusing on the subject), etc.; automatically selecting the transition mode according to the scene switching type, the transition mode includes a dissolve transition for time jumps, a hard cut transition for location jumps, a flash-white transition for perspective changes, and a wipe transition for emotional contrast, and generating corresponding transition prompts and injecting them into the video generation model. Through the rule-based arrangement of shot scale progression and motion mode connection, the switching between shots conforms to the visual grammar of film narrative, avoiding visual jumps and narrative breaks caused by random switching.

[0049] Preferably, the arrangement of shot switching order and transition logic further includes: identifying narrative climax points, transition points, and calm points based on the emotional curve and script conflict structure (a five-stage structure based on the Freytag pyramid model: beginning → rise → climax → fall → ending); allocating a first shot density and first shot duration (e.g., 4-6 shots / minute, 3-5 seconds / shot) to climax points; and allocating a second shot density and second shot duration (e.g., 1-2 shots / minute, 8-12 seconds / shot) to calm points. The first shot density is greater than the second shot density, and the first shot duration is less than the second shot duration. By matching shot density with narrative rhythm, the tension and release of emotional expression are enhanced, giving the generated film and television content a professional narrative rhythm.

[0050] Preferably, the method further includes a character consistency control step: extracting facial feature vectors, body feature vectors, and clothing feature vectors of the target character, and establishing an independent feature vector file for each character; during the video generation stage, injecting the feature vectors of the target character into the AIGC video generation model in the form of adapter weights or low-rank weights. Through the extraction and injection of feature vectors, the character's appearance information is quantified into mathematical vectors and used as constraints on the generation model, ensuring that the character's appearance features in all shots are constrained by the same set of vectors, thereby overcoming the defect of inconsistent appearance of the same character in different shots caused by random sampling in AIGC.

[0051] Preferably, the facial feature vector is used to extract identity feature vectors (based on a 512-dimensional embedding from the ResNet-100 backbone network output) through a face analysis model; the body posture feature vector is used to extract joint coordinate sequences and body proportion parameters (including shoulder width ratio, waist-to-hip ratio, and limb length ratio) through a skeleton detection model; and the clothing feature vector is used to extract clothing texture and color distribution features (HSV color space histogram) through a visual encoder. The feature vectors are stored in a vector database in JSON format, supporting multiple version iterations and rollbacks (using a Git-like branching mechanism, with the main branch representing the current version and the development branch used for testing new looks; before merging, a consistency verification with cross-shot facial similarity greater than 0.85 is required). This multi-version management capability allows for flexible switching of appearance settings for the same character in different episodes or at different stages of the same episode (such as clothing changes or hairstyle changes), while also supporting rapid rollback of historical versions.

[0052] Preferably, the method further includes a scene style unification step: extracting color distribution features (Lab color space global color histogram and dominant color proportion), texture features (contrast, correlation, energy, and homogeneity parameters extracted from the gray-level co-occurrence matrix), and composition features (composition line distribution extracted from Canny edge detection and Hough line detection) from keyframes of reference images or reference videos, generating a style feature vector (256 dimensions); establishing a mapping relationship between the style feature vector and the parameters of the AIGC video generation model, including enabling the corresponding color adjustment mechanism when cool tones dominate, adjusting the guiding coefficient to the corresponding range when high contrast is achieved, and enabling the corresponding image quality enhancement mechanism when cinematic quality is achieved; during the generation of multiple shots in the same scene, locking style-related parameters (including VAE decoder selection, ControlNet preprocessor type, basic sampler type, and global color LUT), allowing only narrative-related parameters (including character action description, camera angle description, and shot type) to change. Through the parameter locking mechanism, all shots in the same scene share the same style configuration, ensuring the consistency of visual style from the underlying parameter level of the generation model, overcoming the deficiency of accurately controlling style by relying solely on prompt text.

[0053] Preferably, the method further includes a parameter optimization step: evaluating the quality of the generated video frames using the image quality index FID and the text-image alignment index CLIP Score. FID is used to evaluate the realism and diversity of the generated images, and CLIP Score is used to evaluate the semantic alignment between the generated images and the text prompts. With the goal of minimizing generation time and maximizing quality score, a parameter search is performed within the parameter space of sampling steps (20 to 50 steps), guidance coefficients (5.0 to 15.0), and resolutions (including 512×512, 768×768, 1024×1024, 1280×720, and 1920×1080), outputting the optimal parameter combination. The generation strategy is dynamically adjusted based on the current computational resource load. The Bayesian optimization algorithm (based on a Gaussian process surrogate model and an expected improved acquisition function) efficiently searches the parameter space, avoiding the computational waste of exhaustive testing for each parameter combination.

[0054] Preferably, the dynamic adjustment generation strategy includes: when the computational resource load is below a first threshold (GPU memory utilization is below 60% and queue waiting time is less than 5 minutes), a standard generation mode (50-step sampling, 1920×1080 resolution, DPM++ 2M Karras sampler) is adopted; when the computational resource load is between the first and second thresholds (GPU memory utilization is 60% to 85% or queue waiting time is 5 to 15 minutes), a balanced generation mode (30-step sampling, 1280×720 resolution, Euler) is adopted. (a) When the computational resource load exceeds the second threshold (memory utilization exceeds 85% or queue waiting time exceeds 15 minutes), the distillation model is enabled to reduce the number of sampling steps (from 50 steps to 4-8 steps), block decoding is enabled to reduce memory usage (the 1024×1024 image is divided into four 512×512 blocks for separate decoding), and the frame interpolation model is enabled to increase the generated frame rate to the target frame rate (first generate 8fps keyframes, then use RIFE or FILM interpolation models to generate intermediate frames to increase to 24fps, reducing the number of keyframes required per shot by 50%). Through a multi-level adaptive switching mechanism, the generation throughput is maximized while ensuring the minimum image quality, enabling the system to maintain stable operation under different load conditions.

[0055] Preferably, the method further includes a quality rollback step: when the quality assessment of the generated video does not reach a preset threshold (FID greater than or equal to 50 or CLIP Score less than or equal to 0.28), the following rollback operations are performed sequentially until the quality meets the standard or the retry limit is reached: adjusting the style description intensity in the structured cue words (e.g., changing the color intensity keyword in the style description from "soft" to "vivid" or adjusting the weight coefficient), re-searching for generation parameter combinations (triggering the parameter search submodule to perform a local re-search centered on the current parameters), and re-initializing the latent space based on the last frame of the previous shot; when the number of retries reaches the limit, the shot is marked as requiring manual processing and skipped to the next shot to continue generation. This rollback mechanism establishes a complete feedback loop from quality assessment to generation parameter adjustment to storyboard cue word correction, overcoming the defect that the failure of a single module causes the entire generation process to be interrupted, and significantly improving the robustness and engineering practicality of the system.

[0056] To address the aforementioned technical issues, this invention also provides an intelligent director scheduling system for AI-powered high-quality film and television content production, comprising: a script semantic parsing engine for semantically parsing the script text and extracting scene elements and emotional curves; an intelligent storyboard-cue word generation module for generating a structured cue word sequence adapted to the AIGC video generation model based on the scene elements and emotional curves; a multi-shot sequence scheduler for arranging shot switching order and transition logic according to the structured cue word sequence, outputting a storyboard sequence with narrative coherence; and a video generation parameter optimizer for dynamically adjusting generation parameters based on the current computing resource load and calling the AIGC video generation model to generate video based on the storyboard sequence. The modules interact with each other through a standardized RESTful API interface. The script parsing results automatically drive storyboard generation, storyboard cue words automatically enter the scheduler for arrangement, the arranged generation request is automatically submitted to the parameter optimizer for execution, and the generated data is fed back to the quality evaluation module in real time, forming a closed loop.

[0057] To address the technical problem of motion jumps and spatial logic breaks caused by random sampling between generated segments, the multi-shot sequence scheduler or the video generation parameter optimizer includes an inter-frame continuity constraint submodule. This submodule uses the last frame information of the current shot as the initial latent space constraint condition for the generation of the next shot. Specifically, after each shot is generated, this submodule maps the generated last frame back to the latent space using a variational autoencoder, extracts its latent space feature vector, and uses this feature vector as a reference for the initial latent space noise in the diffusion process of the next shot. This ensures that the initial state of the next shot remains continuous with the final state of the previous shot in the latent space, thereby maintaining motion coherence and spatial logic consistency during shot transitions in the decoded pixel space. This submodule is the core component of this invention for achieving smooth transitions between shots, overcoming the technical deficiency of existing AIGC tools that can only generate isolated segments and cannot be spliced ​​into a continuous narrative.

[0058] Preferably, the script semantic parsing engine includes: a natural language processing layer for segmenting the script text, named entity recognition, and dependency parsing, identifying scene locations, time settings, character information, prop requirements, and camera movement cues; a sentiment analysis layer for classifying the script text by sentiment, outputting the sentiment intensity value (range 0 to 1) for each scene, and fitting a sentiment curve using a sliding window algorithm; and an element extraction layer for determining narrative climaxes and troughs based on the local extrema of the sentiment curve, and extracting structured scene elements. The natural language processing layer uses the BERT-wwm-ext Chinese pre-trained model for feature extraction, the sentiment analysis layer uses the RoBERTa-wwm-large sentiment classification model, and the element extraction layer uses the BiLSTM-CRF sequence labeling model for structured information extraction.

[0059] Preferably, the intelligent storyboard-cue word generation module includes: a shot language knowledge base for storing shot size rules, camera angle semantic mapping, motion description words, and lighting atmosphere words, stored in JSON Schema format and supporting director-defined extensions; a cue word template engine for filling preset cue word templates according to the scene elements, generating standardized cue words containing subject description, environment description, style description, quality description, and negative cue words, and after filling, performing natural language fluency processing through the T5-small model to generate final cue words that conform to English grammar rules; and a storyboard sequence generator for outputting a storyboard sequence containing shot number, shot type, cue word content, reference frame index, and expected duration based on the scene elements and emotion curve, with the number of shots determined comprehensively based on the number of characters, action complexity, and dialogue density in the scene.

[0060] Preferably, the multi-shot sequence scheduler includes: a narrative rhythm analysis submodule, used to identify narrative climaxes, transitions, and calm points based on the emotional curve and script conflict structure, and assign corresponding shot density and duration weights; a shot switching arrangement submodule, used to generate shot switching order based on shot progression rules and motion mode connection rules, and to perform sequence optimization using a dynamic programming algorithm with the goal of maximizing the naturalness of shot progression and the matching degree of narrative rhythm; and a transition logic generation submodule, used to automatically select transition methods and generate corresponding transition prompts based on scene switching types.

[0061] Preferably, the system further includes a character consistency control module, which includes: a character feature extraction submodule, used to extract facial feature vectors, body feature vectors, and clothing feature vectors of the target character; a feature vector library maintenance submodule, used to establish an independent feature vector file for each character, stored in a Milvus or Pinecone vector database, supporting multiple version iterations and rollbacks; and a generation injection submodule, used to inject the feature vectors of the target character into the AIGC video generation model in the form of adapter weights (IP-Adapter method) or low-rank weights (LoRA method, rank r=16 to 64) during the video generation stage. IP-Adapter converts the facial feature vectors into feature representations in the text alignment space through CLIP image encoder and guides the generation process together with text prompts. LoRA fine-tunes the low-rank weight matrix of the pre-trained model based on the character makeup reference image and loads the weight file during generation to achieve precise control. The two injection methods can be used in combination.

[0062] Preferably, the character feature extraction submodule extracts facial identity feature vectors through a face analysis model (InsightFace model outputs 512-dimensional identity embedding based on ResNet-100 backbone network), extracts body posture feature vectors through a skeleton detection model (OpenPose model extracts 18 key point coordinate sequences and body posture proportion parameters), and extracts clothing texture and color distribution feature vectors through a visual encoder (ResNet-50 visual encoder extracts texture features and HSV color space histogram); the feature vector library maintenance submodule stores feature vectors based on a vector database and uses a Git-like branching mechanism for version management; the generation and injection submodule converts low-rank weights to a format compatible with the distillation model before injection when acceleration mode is enabled, to ensure that character control capabilities are not lost when accelerating generation using LCM or SDXL-Turbo distillation models.

[0063] Preferably, the system further includes a scene style unification module, which comprises: a style reference extraction submodule, used to extract color distribution features (Lab color space global color histogram), texture features (grayscale co-occurrence matrix contrast, correlation, energy, homogeneity) and composition features (line distribution extracted by Canny edge detection and Hough line detection) of keyframes of reference images or reference videos, generating a 256-dimensional style feature vector; a style vector mapping submodule, used to establish a mapping relationship between the style feature vector and the parameters of the AIGC video generation model, the mapping rules including the color adjustment mechanism corresponding to the color tone, the guidance coefficient adjustment range corresponding to the contrast level, and the enhancement mechanism corresponding to the image quality, the mapping table is stored in YAML format and supports director-customized extensions; and a generation parameter locking submodule, used to lock style-related parameters (VAE decoder selection, ControlNet preprocessor type, basic sampler type, global color LUT) through a generation configuration template during the generation of multiple shots in the same scene, allowing only narrative-related parameters (character action description, camera angle description, shot type) to change, and if abnormal changes in style parameters are detected, a style consistency alarm is triggered and the system automatically rolls back to the locked value.

[0064] Preferably, the video generation parameter optimizer includes: a quality assessment submodule, used to evaluate the quality of the generated video frames through FID (based on Inception-v3 feature extraction, used to evaluate the realism and diversity of the generated images) and CLIP Score (based on the CLIPViT-B / 32 model to calculate the semantic alignment between text prompts and generated images); and a parameter search submodule, used to minimize generation time and maximize quality score by considering the number of sampling steps (20 to 50 integer steps), guidance coefficients (5.0 to 15.0 consecutive floating-point numbers), resolution (512×512, 768×768, 1024×1024, 1280×720, 1920×1080 enumerated values), and sampler type (Euler a, DPM++ 2M). The parameter search is performed within the parameter space of Karras and DDIM enumeration values ​​using Bayesian optimization. After 10 to 20 iterations, the Pareto optimal parameter combination is output. The adaptive adjustment submodule is used to dynamically adjust the generation strategy based on the current computing resource load. The resource monitoring daemon automatically triggers mode switching based on NVIDIA-SMI and queue length detection. When the computing resource load is higher than the threshold, the distillation model is enabled to reduce the number of sampling steps, the block decoding is enabled to reduce the video memory usage, and the frame interpolation model is enabled to improve the generation frame rate.

[0065] Preferably, the video generation parameter optimizer further includes a generation quality rollback submodule. This submodule is used to perform the following rollback operations sequentially until the quality meets the standard or the retry limit is reached when the quality assessment of the generated video fails to meet a preset threshold (FID greater than or equal to 50 or CLIP Score less than or equal to 0.28): sending a style intensity adjustment instruction to the intelligent storyboard-cue word generation module to modify the cue words (e.g., adjusting "blurry" in the negative cue words to "extremely blurry, out of focus" to enhance the constraint strength); re-triggering the parameter search submodule to search for new parameter combinations (using the current failed parameter as the center, reducing the search radius, and performing a local re-search); re-initializing the latent space based on the last frame of the previous shot; and marking the shot as requiring manual processing and skipping to the next shot to continue generation after the retry count reaches the limit, thus avoiding single-point failure blocking the entire generation pipeline.

[0066] The structure and working process of the present invention will be further described below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0067] like Figure 1As shown, the system of this invention includes a script semantic parsing engine, an intelligent storyboard-prompt generation module, a character consistency control module, a scene style unification module, a multi-shot sequence scheduler, and a video generation parameter optimizer. These modules interact with each other via standardized RESTful API interfaces, forming a complete automated pipeline from front-end script input to back-end video generation.

[0068] Taking the production of a specific 20-episode period drama AI short series as an example, each episode is approximately 3 minutes long and contains about 15 scenes and 25 shots. The complete working process of this invention is described in detail below with reference to the accompanying drawings.

[0069] Phase 1: Script Semantic Analysis

[0070] like Figure 2 As shown, the script text is first obtained and then input into the script semantic parsing engine. This engine contains three processing layers:

[0071] Natural Language Processing Layer: Employing a pre-trained Chinese model based on BERT-wwm-ext (Whole Word Masking), the script text undergoes word segmentation, Named Entity Recognition (NER), and Dependency Parsing. Entity recognition types include: Person (PER), Location (LOC), Time (TIME), and Prop (PROP). Dependency parsing extracts subject-verb-object structures, identifying character actions and their recipients. Processing results are output in CoNLL format. For example, in the script's description, "The male protagonist slowly turned around and looked out the window at the Huangpu River," the system identifies the character entity "male protagonist" (type PER) and the location entity "Huangpu River" (type LOC). Through dependency parsing, it extracts the subject-verb-object structure and identifies the actions as "slowly turned around" and "looked out the window." The processing results are then output to the sentiment analysis layer in CoNLL format.

[0072] The sentiment analysis layer employs a RoBERTa-wwm-large-based sentiment classification model to categorize the text for each scene into three sentiment categories: positive, neutral, and negative, outputting a sentiment intensity value ranging from 0 to 1. For example, the sentiment intensity value for a parting scene is 0.85, for a reunion scene it is 0.92, and for a transitional dialogue scene it is 0.45. Then, a sliding window algorithm is used, with a window size of three scenes, to smoothly fit the sentiment value sequence. When the sentiment intensity value at the center of the window is simultaneously greater than its left and right adjacent points, it is marked as a local maximum, i.e., a narrative climax; when it is simultaneously less than its left and right adjacent points, it is marked as a local minimum, i.e., a narrative trough. For example, in a certain episode, scene 5 is a climax with a sentiment intensity of 0.88, allocated with high-density shots (5 close-ups per minute, each shot lasting 3 seconds); scene 12 is a trough with a sentiment intensity of 0.18, allocated with low-density long shots (1 long shot per minute, lasting 10 seconds).

[0073] The element extraction layer employs a hybrid approach combining rule-based templates and a BiLSTM-CRF sequence labeling model to extract structured elements from the script text. These elements include scene locations such as "Shanghai Bund," time settings such as "1940s," character actions such as "slowly turning around," character dialogue such as "Are you really leaving?", prop requirements such as "old-fashioned telephone," environmental atmosphere such as "continuous rain," and camera movement cues such as "the camera slowly zooms in." The extraction results are output in JSON format, containing scene numbers, locations, times, a list of characters and their respective descriptions of actions, emotions, and costumes, a list of props, environmental atmosphere descriptions, emotion intensity values, and camera movement cues. The semantic parsing engine transmits the parsed structured data to the intelligent storyboard-prompt generation module via an API interface.

[0074] After adopting this engine, the script analysis and prompt generation time for a 20-episode AI short drama (3 minutes per episode, about 15 scenes) was reduced from 40-60 hours by manual writing to 2.5 hours, and the alignment between prompts and script intent (based on GPT-4 semantic similarity assessment) increased from 72% by manual writing to 89%.

[0075] Phase Two: Intelligent Storyboarding and Prompt Generation

[0076] like Figure 3 As shown, the intelligent storyboard-cue word generation module receives structured scene data from the script semantic parsing engine. This module includes three components: a shot language knowledge base, a cue word template engine, and a storyboard sequence generator.

[0077] The camera language knowledge base stores various camera rules adapted to the AIGC video generation model in JSON Schema format: Shot size rules define close-ups to emphasize facial expressions, medium shots to show upper body movements, long shots to show character interactions, wide shots to show environmental relationships, and long shots to show grand scenes; Camera angle semantic mapping defines eye-level shot, high angle shot, and low angle shot; Motion description lexicon includes dolly in, dolly out, pan, truck, follow shot, and static shot; Lighting and atmosphere lexicon includes harsh lighting, soft lighting, backlight, side lighting, and top lighting.

[0078] The prompt template engine generates prompt templates by filling in pre-defined AIGC videos with structured scene elements. The main description template includes character names, clothing details, action states, facial expressions, prop holding, and body features; the environment description template includes scene location, time setting, weather and lighting, background elements, and depth of field; the style description template includes art style, reference director, color scheme, and image quality; the quality description template includes resolution, level of detail, rendering quality, and dynamic smoothness; and the negative prompt template includes undesirable features such as blurriness, distortion, redundant limbs, inconsistent lighting, and low quality. After filling in the prompts, the T5-small model is used for natural language smoothing to generate final prompts that conform to English grammar.

[0079] Storyboard Sequence Generator: Based on script scene elements, emotional curves, and narrative rhythm, the generator outputs a storyboard sequence including shot number, shot type, cue words, reference frame index, and expected duration. The generator determines the number of shots based on the number of characters, action complexity, and dialogue density in each scene: 2-3 shots for single-person scenes, 3-5 shots for two-person scenes, and 5-8 shots for three or more-person scenes; 1-2 shots for simple actions and 3-4 shots for complex actions; and 1 shot for every 50 words of dialogue. Shot type is determined by emotional intensity value: greater than 0.8 uses close-ups to emphasize emotion, 0.5-0.8 uses medium or close-up shots to show interaction, and less than 0.5 uses wide or long shots to establish the environment. Movement method is selected based on keywords in the character's action description: push-in shots for "slowly," panning shots for "looking around," following shots for "running," and fixed shots for dialogue scenes. Finally, the storyboard sequence generator outputs a complete storyboard sequence containing shot number, shot type, cue words, reference frame index, and expected duration. The sequence is then optimized using a dynamic programming algorithm, with the state defined as the current shot type and the decision defined as the next shot type. The goal is to maximize the naturalness of the shot progression and the match between the shot type and the narrative rhythm.

[0080] After adopting this module, the time for generating storyboards and prompts for a single AI short drama (3 minutes, about 25 shots) has been reduced from 3-4 hours for manual writing to 15 minutes, with a prompt generation pass rate of 92%.

[0081] Phase 3: Role Consistency Control and Scene Style Unification

[0082] like Figure 4 As shown, the character consistency control module receives the character makeup reference image. The character feature extraction submodule extracts multi-dimensional features using three models: the InsightFace face analysis model extracts a 512-dimensional facial identity feature vector based on the ResNet-100 backbone network, which is robust to changes in expression, posture, and lighting; the OpenPose skeleton detection model extracts 18 joint coordinate sequences and calculates body posture parameters such as shoulder-width ratio, waist-to-hip ratio, and limb length ratio; the ResNet-50 visual encoder extracts clothing texture, HSV color space histogram distribution, and style feature vectors. All feature vectors are stored in JSON format, including, for example, character ID, character name, facial feature vector array, body posture parameters, and clothing feature data.

[0083] The feature vector library maintenance submodule creates an independent feature vector file for each character, stored in the Milvus vector database. It employs a Git-like branching mechanism for version management: the main branch represents the currently used version, while the development branch is used for testing new looks. Before merging, a consistency verification is required, meaning the cross-camera facial similarity must be greater than 0.85. When the same character needs to change clothing or hairstyle in different episodes, the system creates a new version branch instead of overwriting the original features, supporting quick rollback of historical versions.

[0084] The injection submodule injects the feature vector of the target character into the AIGC video generation model during the video generation stage, using either adapter weights or low-rank weights. The IP-Adapter injection method uses the character's facial feature vector as image conditional input, converting it into a feature representation in text alignment space via the CLIP image encoder, which, along with text prompts, guides the generation process. The LoRA injection method fine-tunes the low-rank weight matrix of the pre-trained model based on the character's makeup reference image, setting the rank r to 16 to 64, and loading a LoRA weight file in .safetensors format during generation. The two injection methods can be used in combination; IP-Adapter controls facial consistency, while LoRA controls clothing and overall style consistency.

[0085] After adopting this module, the average facial similarity of characters across shots (based on FaceNet cosine similarity) increased from 0.62 to 0.91, the accuracy of character clothing consistency recognition (based on manual annotation verification) increased from 45% to 88%, and the audience character recognition score (out of 10) increased from 4.8 to 7.9.

[0086] like Figure 5 As shown, the scene style unification module receives the scene reference image. The style reference extraction submodule calculates the global color histogram and extracts color distribution features by using the Lab color space and the proportion of the dominant color tone. It extracts contrast, correlation, energy, and homogeneity parameters as texture features by using the gray-level co-occurrence matrix. It extracts the distribution of composition lines as composition features by using Canny edge detection and Hough line detection. The modules are then combined to generate a 256-dimensional style feature vector.

[0087] The style vector mapping submodule establishes a mapping table between style feature vectors and AIGC video generation model parameters, stored in YAML format. Mapping rules include: Cool tone dominance (Lab b star less than 0) activates the corresponding color adjustment mechanism; warm tone dominance (Lab b star greater than 0) activates the corresponding warm tone mechanism; high contrast (GLCM contrast greater than 0.5) adjusts the guiding coefficient to the range of 10-12; low contrast (GLCM contrast less than 0.3) adjusts the guiding coefficient to the range of 6-8; cinematic quality (texture energy greater than 0.7) activates the corresponding image quality enhancement mechanism; and sharp digital quality (texture energy less than 0.4) activates the corresponding sharpness mechanism.

[0088] The generation parameter locking submodule locks style-related parameters during the generation of multiple shots in the same scene by generating configuration templates. These parameters include VAE decoder selection, ControlNet preprocessor type, base sampler type, and global color LUT. Only narrative-related parameters such as character action descriptions, camera angle descriptions, and shot type are allowed to change. If an abnormal change in style parameters is detected, such as an unexpected switch of the VAE, the system triggers a style consistency alert and automatically rolls back to the locked values.

[0089] After adopting this module, the color difference of shots in the same scene (based on Euclidean distance in Lab color space) was reduced by 67%, the visual style consistency score (blindly evaluated by 10 viewers, out of 10) increased from 5.2 to 8.1, and the perception rate of style jumps in the same scene decreased from 38% to 8%.

[0090] Phase 4: Multi-camera sequence scheduling

[0091] like Figure 6 As shown, the multi-shot sequence scheduler receives the storyboard sequence from the intelligent storyboard-prompt generation module and the constraints from the character consistency control module and the scene style unification module, and then arranges the narrative.

[0092] The narrative rhythm analysis submodule identifies narrative climaxes, transition points, and calm points based on the emotional curve and the Freytag pyramid script conflict structure model (comprising five stages: beginning, rise, climax, fall, and ending). Climaxes are located at 60% to 80% of the script, with an emotional intensity greater than 0.8, and are allocated high-density shots (4 to 6 shots per minute), primarily close-ups and extreme close-ups, with each shot lasting 3 to 5 seconds to enhance emotional impact. Transition points have an emotional intensity of 0.4 to 0.6, and are allocated medium shots, with each shot lasting 5 to 8 seconds. Calm points have an emotional intensity less than 0.4, and are allocated low-density long shots (1 to 2 shots per minute), primarily wide shots and extreme close-ups, with each shot lasting 8 to 12 seconds, to establish environmental information and narrative background.

[0093] The shot transition choreography submodule generates the shot transition sequence based on shot size change rules and motion transition rules. Shot size progression rules include a zoom-in sequence (long shot → wide shot → medium shot → close-up → extreme close-up) for building emotional intensity, and a zoom-out sequence (extreme close-up → medium shot → wide shot → long shot) for releasing emotional tension. Shot size jump rules allow direct transitions from extreme close-up to long shot to create emotional contrast, but prohibit direct transitions from medium shot to medium shot to avoid visual monotony. Motion transition rules define a push-in shot followed by a stationary shot for stabilizing emotions, a pull-out shot followed by a panning shot for expanding the field of view, a follow shot followed by a stationary shot for stopping movement, and a panning shot followed by a push-in shot for focusing on the subject. The choreography result is output as a JSON sequence, including shot number, shot size, motion transition, duration, cue word, and transition type to the next shot.

[0094] The transition logic generation submodule automatically selects the transition method based on the scene switching type. For time transitions such as "three years later," a dissolve transition is used, generating the transition prompt "smooth dissolve transition, time lapse effect"; for location transitions such as "switching to the office," a hard cut transition is used, generating "clean hard cut, new location introduction"; for perspective changes such as "switching to the villain's perspective," a flash white transition is used, generating "flash white transition, perspective shift"; and for emotional contrasts such as switching from comedy to tragedy, a wipe transition is used, generating "diagonal wipe transition, mood contrast." The transition prompts are subsequently injected into the inpainting region of the video generation model or generated as independent transition segments.

[0095] After adopting this scheduler, the narrative coherence score of the generated segments (based on the GPT-4 narrative analysis API, evaluated from three dimensions: plot logic, emotional progression, and pacing control, with a maximum score of 10) increased from 6.1 to 8.4, the audience completion rate (based on A / B testing, sample size N=2000) increased from 32% to 58%, and the narrative jump perception rate decreased from 42% to 15%.

[0096] Phase 5: Inter-frame continuity constraints and video generation

[0097] After the above scheduling and arrangement, the storyboard sequence is submitted to the video generation parameter optimizer for generation. This invention specifically introduces an inter-frame continuity constraint mechanism during the generation process:

[0098] Specifically, such as Figure 6As shown, the inter-frame continuity constraint submodule is built into the multi-shot sequence scheduler or video generation parameter optimizer. After the first shot is generated, the system maps the generated last frame image back to the latent space through the encoder part of the variational autoencoder, extracts its latent space feature vector (i.e., the latent space state at the end of the diffusion process denoising), and uses this feature vector as the initial latent space reference for the diffusion process of the second shot, instead of using random Gaussian noise as the initial state. Specifically, during the diffusion process of the second shot, the latent space feature vector of the last frame of the first shot is used as the initial latent space variable. A small amount of Gaussian noise is added to this, and then the reverse denoising process begins, so that the initial image state of the second shot remains continuous with the final state of the first shot in the latent space. For the generation of subsequent shots, the above process is repeated, so that the generation of each shot evolves based on the latent space of the last frame of the previous shot.

[0099] This inter-frame constraint mechanism ensures the physical continuity of the scene content during shot transitions—when the camera switches from a close-up of character A's face to a medium shot of character A's action, the facial orientation, expression, clothing folds, and spatial position of character A in the latter shot maintain a natural transition with the last frame of the former, avoiding common AI video problems such as character teleportation, sudden changes in expression, and jumps in the layout of scene elements. Simultaneously, a reference frame index is recorded in the storyboard sequence to identify the last frame information of the preceding shot that each shot relies on.

[0100] Phase 6: Generation Parameter Optimization and Quality Regression

[0101] After receiving the storyboard sequence, the video generation parameter optimizer executes the parameter optimization process:

[0102] The quality assessment submodule evaluates the realism and diversity of generated video frames based on Inception-v3 feature extraction using FID (Frame Identifier). An FID less than 50 is considered high quality, 50 to 100 is medium quality, and greater than 100 is low quality. The CLIP Score is used to calculate the semantic alignment between text prompts and generated images based on the CLIP ViT-B / 32 model. A score greater than 0.28 is considered good alignment, 0.20 to 0.28 is considered partial alignment, and less than 0.20 is considered alignment failure. The combined criterion for both metrics is that an FID less than 50 and a CLIP Score greater than 0.28 are considered acceptable.

[0103] The parameter search submodule aims to minimize generation time and maximize quality score, employing a Bayesian optimization algorithm to search within the parameter space. The parameter space includes discrete integers with sampling steps ranging from 20 to 50, continuous floating-point numbers with guiding coefficients ranging from 5.0 to 15.0, discrete enumerated values ​​with resolutions ranging from 512×512 to 1920×1080, and discrete enumerated values ​​for sampler types Euler a, DPM++2M Karras, and DDIM. Bayesian optimization iterates based on a Gaussian process surrogate model and the desired improved acquisition function, outputting the Pareto optimal parameter combination after 10 to 20 iterations.

[0104] The adaptive adjustment submodule automatically triggers policy switching based on NVIDIA-SMI and queue length detection through the resource monitoring daemon. When GPU memory usage is below 60% and queue waiting time is less than 5 minutes, the standard generation mode is used, i.e., 50 sampling steps, 1920×1080 resolution, and DPM++ 2M Karras sampler. When GPU memory usage is between 60% and 85% or queue waiting time is between 5 and 15 minutes, the balanced mode is used, i.e., 30 sampling steps, 1280×720 resolution, and Euler a sampler. When GPU memory usage is above 85% or queue waiting time is more than 15 minutes, the acceleration mode is enabled: LCM or SDXL-Turbo distillation model is enabled to reduce the number of sampling steps from 50 to 4 to 8 steps, Tile VAE block decoding is enabled to divide the 1024×1024 image into 4 blocks of 512×512 and decode them separately to reduce memory usage, and RIFE or FILM frame interpolation model is enabled to first generate 8fps keyframes and then generate intermediate frames through interpolation to increase to 24fps, reducing the number of keyframes required to generate per shot by 50%.

[0105] When accelerated mode is enabled, the generation injection submodule of the character consistency control module converts the loaded LoRA low-rank weights into a format compatible with LCM or SDXL-Turbo distillation models before injection, ensuring that character appearance control is not lost during accelerated generation.

[0106] After adopting this optimizer, while ensuring the quality thresholds of FID < 50 and CLIP Score > 0.28, the average generation time of a single shot (5 seconds, 24fps, 120 frames in total) is reduced from 8 minutes to 3.5 minutes, the GPU resource utilization rate is increased from 45% to 78%, and the daily production capacity is increased from 120 shots to 280 shots under the same resource conditions.

[0107] The quality rollback submodule performs quality checks on the results after each generation. When the generated video frame's FID is greater than or equal to 50 or its CLIP Score is less than or equal to 0.28, the rollback process is triggered: First, a style intensity adjustment instruction is sent to the intelligent storyboard-cue word generation module, enhancing the relevant description in the negative cue word from "blurry" to "extremely blurry, out of focus" to strengthen the constraint; if the retry still fails, the parameter search submodule is retried, narrowing the search radius around the currently failed parameters for a local research; if it still fails, the initial latent space state of the current shot is reinitialized based on the latent space characteristics of the previous shot's last frame, and a third retry is performed. When the number of retries reaches the maximum of 3, the shot is marked as requiring manual processing and skipped to the next shot to continue generation, avoiding single-point failure blocking the entire generation pipeline. Testing shows that the system's self-healing success rate for this rollback mechanism is approximately 76%.

[0108] Through the collaborative work of the above stages, this embodiment completes a fully automated production process from script text input to multi-camera narrative video output, verifying the effectiveness and advancement of the present invention.

[0109] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the invention without departing from the principles and spirit of the invention, and all such changes should fall within the protection scope of the claims of the present invention.

Claims

1. A method for intelligent director scheduling in the production of high-quality AI-generated film and television content, characterized in that, Includes the following steps: Obtain the script text, perform semantic analysis on the script text, and extract scene elements and emotional curves; Based on the scene elements and emotion curves, a structured cue word sequence adapted to the AIGC video generation model is generated, and the shot switching order and transition logic are arranged according to the structured cue word sequence to output a storyboard sequence with narrative coherence. The AIGC video generation model is invoked according to the storyboard sequence to generate video. During the generation process, the last frame information of the current shot is used as the initial latent space constraint condition for the generation of the next shot, so as to maintain the motion continuity and the consistency of the spatial logic of the picture when switching shots.

2. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 1, characterized in that, The semantic parsing includes: segmenting the script text, identifying named entities, and performing dependency parsing to identify scene locations, time settings, character information, prop requirements, and camera movement cues; classifying the script text for emotion, outputting the emotion intensity value for each scene, and fitting the emotion curve using a sliding window algorithm; identifying local extreme points of the emotion curve, and determining the narrative climax and trough based on the local extreme points.

3. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 1, characterized in that, The generated structured prompt word sequence includes: based on the shot size rules, camera angle semantic mapping, movement mode description words and lighting atmosphere words in the shot language knowledge base, filling the preset prompt word template according to the scene elements, generating standardized prompt words containing subject description, environment description, style description, quality description and negative prompt words; in the prompt word template, the subject description includes character name, action state, facial expression characteristics, clothing details and prop holding information, the environment description includes scene location, time setting, weather lighting, background elements and depth of field information, and the style description includes art style, reference director, color tendency and image quality information.

4. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 1, characterized in that, The arrangement of shot switching order and transition logic includes: generating shot switching order based on shot scale progression rules and motion mode connection rules, wherein shot scale progression rules include zoom-in sequences and zoom-out sequences, and motion mode connection rules include the continuity relationship between different types of shot motion; automatically selecting the transition mode according to the scene switching type, wherein the transition mode includes dissolve transition, hard cut transition, flash white transition and wipe transition, and generating corresponding transition prompt words.

5. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 4, characterized in that, The arrangement of shot switching order and transition logic also includes: identifying narrative climax points, transition points and calm points based on the emotional curve and script conflict structure, allocating a first shot density and a first shot duration to the climax points, and allocating a second shot density and a second shot duration to the calm points, wherein the first shot density is greater than the second shot density and the first shot duration is less than the second shot duration.

6. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 1, characterized in that, It also includes a character consistency control step: extracting the facial feature vector, body feature vector, and clothing feature vector of the target character, and establishing an independent feature vector file for each character; in the video generation stage, the feature vector of the target character is injected into the AIGC video generation model in the form of adapter weights or low-rank weights.

7. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 6, characterized in that, The facial feature vector is used to extract identity feature vectors through a face analysis model. The body posture feature vector is used to extract joint point coordinate sequences and body posture proportion parameters through a skeleton detection model. The clothing feature vector is used to extract clothing texture and color distribution features through a visual encoder. The feature vectors are stored in a vector database in JSON format and support multiple version iterations and rollbacks.

8. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 1, characterized in that, It also includes a scene style unification step: extracting color distribution features, texture features, and composition features from keyframes of reference images or reference videos to generate style feature vectors; establishing a mapping relationship between the style feature vectors and the parameters of the AIGC video generation model; and locking style-related parameters during the generation of multiple shots in the same scene, allowing only narrative-related parameters to change.

9. The intelligent director scheduling method for AI-driven high-quality film and television content production according to claim 1, characterized in that, It also includes a parameter optimization step: evaluating the quality of the generated video frames using image quality metrics and text-image alignment metrics; searching for parameters within the sampling steps, guidance coefficient, and resolution parameter space with the goal of minimizing generation time and maximizing quality score, and outputting the optimal parameter combination; and dynamically adjusting the generation strategy based on the current computing resource load.

10. An intelligent director scheduling system for AI-driven high-quality film and television content production, characterized in that: include: The script semantic parsing engine is used to semantically parse the script text and extract scene elements and emotional curves; The intelligent scene-prompt generation module is used to generate a structured prompt sequence adapted to the AIGC video generation model based on the scene elements and emotion curves. A multi-shot sequence scheduler is used to arrange the shot switching order and transition logic according to the structured cue word sequence, and output a storyboard sequence with narrative coherence. A video generation parameter optimizer is used to dynamically adjust the generation parameters based on the current computing resource load, and to call the AIGC video generation model to generate video based on the storyboard sequence; The multi-shot sequence scheduler or the video generation parameter optimizer includes an inter-frame continuity constraint submodule. The inter-frame continuity constraint submodule is used to use the last frame information of the current shot as the initial latent space constraint condition for the generation of the next shot, so as to maintain the motion continuity and spatial logic consistency during shot switching.

Images