A training-free movie-level long video generation method and system

Abstract

This application provides a training-free method and system for generating cinematic-quality long videos, relating to the field of video generation technology. This application obtains predefined control parameters and user scripts from a control parameter library, and generates a structured shot triplet through the collaborative use of two large language models. Based on this triplet, a storyboard is created using a camera-aware routing mechanism to generate a keyframe sequence. Then, combining the keyframe sequence, camera parameters from the predefined control parameters, and scene descriptions, independent video segments are generated using keyframe-to-video interpolation technology. Finally, seamless transition processing guided by trajectory is applied to the independent video segments to obtain transition segments. First-order continuous interpolation technology is used to stitch together the independent video segments and the transition segments to ultimately obtain a cinematic-quality long video. The entire process requires no training to generate cinematic-quality long videos, enabling accurate generation of multi-dimensional, controllable, and coherent cinematic-quality long videos without any training.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of video generation, in particular to a movie-level long video generation method and system without training. BACKGROUND

[0002] The movie-level long video generation method without training can quickly convert creative ideas into high-quality video content, has broad application prospects in the fields of film and television creation, advertisement production, short video production, etc., and can greatly reduce the technical threshold and time cost of professional video production.

[0003] At present, long video generation technologies without training mostly rely on end-to-end text-to-video models to directly generate video content by inputting text prompts. Such methods do not require complex model training in advance and can quickly respond to generation requirements, so they are widely used in the industry.

[0004] However, the existing such technologies are difficult to achieve movie-level fine-grained control of camera trajectories, lens types, etc., and the generated multi-lens videos often have the problem of insufficient coherence, which cannot meet the requirements of professional film and television production for precise control of content and visual fluency. Therefore, there is a technical problem in the prior art that the control accuracy and coherence of movie-level long video generation are difficult to balance. SUMMARY

[0005] The purpose of the present application is to provide a movie-level long video generation method and system without training to solve the problem that the control accuracy and coherence of movie-level long video generation are difficult to balance in the prior art.

[0006] To solve the above technical problems, in a first aspect, the present application provides a movie-level long video generation method without training, comprising: obtaining predefined control parameters and a user script from a control parameter library, generating a script by a dual large language model working in cooperation according to the control parameters and the user script, and obtaining a structured lens triplet; based on the structured lens triplet, generating a keyframe sequence by a storyboard production through a camera perception routing mechanism; based on the keyframe sequence, camera parameters and scene descriptions in the predefined control parameters, generating an independent video segment by a keyframe-to-video interpolation technology; based on the independent video segment, generating a transition segment by a seamless transition processing guided by a trajectory, and splicing the independent video segment and the transition segment by a first-order continuous interpolation technology to obtain a movie-level long video.

[0007] Optionally, based on the keyframe sequence, camera parameters and scene descriptions in the predefined control parameters, an independent video segment is generated by a keyframe-to-video interpolation technology, comprising: Extract adjacent keyframe pairs from the keyframe sequence, combine them with the camera parameters, and convert the adjacent keyframe pairs and camera parameters into spatiotemporal control signals through encoding processing; Using the spatiotemporal control signal as a constraint, the image changes between adjacent keyframe pairs are deduced through the Hero AI interpolation algorithm to generate a continuous intermediate frame sequence. Geometric stability verification is performed on the intermediate frame sequence. By detecting the pixel displacement between frames, the matching degree of the main body contour, and the consistency of background elements, inconsistencies between frames are automatically identified and repaired in a targeted manner. The inconsistencies between frames include main body drift and background abrupt changes. By using keyframe-to-video interpolation technology, the repaired intermediate frame sequence is integrated to form a visually coherent and content-complete independent video segment.

[0008] Optionally, based on the control parameters and the user script, a script is generated through the collaborative work of two large language models to obtain a structured shot triplet, including: The predefined control parameters include movie type tags, number of shots, and camera parameters; the dual language models include a first language model and a second language model. Using the first large language model as a scene generator, a structured scene description including lighting, location, subject, action, and crowd density is generated based on the movie type label, number of shots, and core plot of the user script. Using the second language model as a cinematographer, based on the abstract camera instructions corresponding to the camera parameters and combined with the structured scene description, the abstract camera instructions are converted into precise visual descriptions; Based on the structured scene description and the precise visual description, a structured shot triad is derived, which includes the initial shot, the camera parameters, and the final shot.

[0009] Optionally, based on the structured shot triplet, a storyboard is created using a camera-aware routing mechanism to generate a keyframe sequence, including: Based on the structured scene description in the structured lens tripartite and combined with the image composition requirements corresponding to the camera parameters, a high-fidelity starting image is generated. Based on the high-fidelity starting frame, and combined with the precise visual description and camera motion trend in the structured lens triplet, continuous keyframes are generated. The dynamic routing algorithm using the camera-aware routing mechanism selects the optimal model based on the camera motion type corresponding to the camera parameters in the structured lens triplet to generate a high-fidelity keyframe sequence, where the static camera corresponds to the Gemini model and the panning camera corresponds to the Tongyi Qianwen model.

[0010] Optionally, based on the independent video segments, a transition segment is generated through trajectory-guided seamless transition processing, and the independent video segments and the transition segment are spliced ​​together using first-order continuous interpolation technology to obtain a cinematic long video, including: The third-generation collaborative tracker performs feature point matching on the frames of adjacent independent video segments, and establishes sparse corresponding points between adjacent frames. The accuracy of the sparse corresponding points is controlled within ±0.5 pixels. Adaptive window truncation is applied to the frame sequences of adjacent independent video segments to dynamically identify and delete static frames with a motion threshold of less than 0.1 pixels / frame. Based on the sparse corresponding points, the correlation between the positions of the frames is determined. Using parameters with values ​​ranging from 0 to 1 as variables, and combining the initial position and initial velocity of the independent video segment with the termination position and termination velocity of the adjacent segments, a cubic Hermit spline path is constructed, and a smooth trajectory is obtained by trajectory fitting. Guided by the smooth trajectory, a transition segment connecting adjacent independent video segments is generated; The independent video segments and the corresponding transition segments are spliced ​​frame by frame using first-order continuous interpolation technology to form a coherent and complete cinematic long video.

[0011] Optionally, it also includes: Before generating the cinematic-quality long video, a balanced, high-quality video dataset of 5,000 videos is pre-built; The method described in claim 1 adopts a four-stage decoupling architecture, which corresponds to script generation action, storyboard production action, video clip generation action and seamless transition and splicing action respectively. The components corresponding to each stage action can be upgraded independently without retraining the entire system. The cinematic long video supports multi-dimensional joint control of 17 camera movements and 15 film genres, with an average duration of 12 seconds and supports 3-lens switching.

[0012] Optionally, each stage of the method supports manual intervention and conforms to film and television production standards; the camera-aware routing mechanism is a model routing mechanism for camera motion type perception; the application process of the keyframe-to-video interpolation technology includes the generation of dual-condition constraints of keyframes and camera commands; the trajectory-guided seamless transition processing includes trajectory-guided smooth spatiotemporal interpolation.

[0013] Secondly, this application provides a training-free cinematic long video generation system, including: The script generation module is used to obtain predefined control parameters and user scripts from the control parameter library, and generate scripts based on the control parameters and user scripts through the collaborative work of two major language models, resulting in structured shot triples. The storyboard creation module is used to create a storyboard based on the structured shot triplet and through a camera-aware routing mechanism, generating a keyframe sequence. The video clip generation module is used to generate independent video clips based on the keyframe sequence, camera parameters in the predefined control parameters, and scene description, using keyframe-to-video interpolation technology. A seamless transition module is used to generate a transition segment based on the independent video segment through trajectory-guided seamless transition processing, and to splice the independent video segment and the transition segment using first-order continuous interpolation technology to obtain a cinematic long video.

[0014] Thirdly, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor is configured to execute the computer program to implement the steps of the training-free cinematic long video generation method as described in the first aspect above.

[0015] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the training-free cinematic long video generation method described in the first aspect above.

[0016] The training-free cinematic long video generation method provided in this application obtains predefined control parameters and user scripts from a control parameter library, and generates structured shot triples through the collaborative generation of two major language models. This clearly defines the core elements of the video and camera control requirements, providing a precise basis for subsequent generation. By creating storyboards and generating keyframe sequences based on structured shot triples and camera-aware routing mechanisms, the method ensures the adaptability of keyframes to scenes and camera commands, laying the foundation for high-quality videos. By combining keyframe sequences, camera parameters, and scene descriptions, and using keyframe-to-video interpolation technology to generate independent video segments, the method achieves dual constraints of keyframes and camera commands, ensuring the integrity of the segment content. By performing trajectory-guided seamless transition processing and splicing on independent video segments, the method eliminates shot transition gaps and achieves smooth and coherent long videos.

[0017] Furthermore, adjacent keyframe pairs in the keyframe sequence are extracted and converted into spatiotemporal control signals by combining camera parameters. These signals are then used as constraints to generate intermediate frame sequences via AI interpolation. After geometric stability verification to correct inter-frame inconsistencies, keyframe-to-video interpolation techniques are used to integrate these sequences into independent video clips. Through the synergistic effect of dual-constraint interpolation and stability verification, issues such as subject drift and abrupt background changes are effectively avoided, ensuring that the generated independent video clips are visually coherent and accurate, meeting the quality requirements of cinematic video clips. Attached Figure Description

[0018] To more clearly illustrate the technical solutions of the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 A flowchart illustrating a training-free method for generating cinematic long videos provided in this application embodiment; Figure 2 A flowchart illustrating a training-free method for generating cinematic long videos provided in this application. Figure 3 This is a schematic diagram of the structure of a training-free cinematic long video generation system provided in an embodiment of this application. Detailed Implementation

[0020] In the field of cinematic long video generation, training-free generation technology has attracted much attention because it can quickly respond to creative needs and reduce production costs. However, the existing technical path that relies on end-to-end text-to-video models has obvious shortcomings: on the one hand, it is difficult to achieve fine-grained control over camera trajectory, shot type, etc., and cannot accurately reproduce creative intentions; on the other hand, the coherence of multi-shot videos is insufficient, and some datasets have problems such as imbalanced annotation and high noise, which further affect the generation quality and make it difficult to meet the requirements of professional film and television creation.

[0021] To address the aforementioned issues, this application proposes a training-free, cinematic-quality long-form video generation method that achieves precise control and coherent generation through a four-stage progressive process. First, based on control parameters and the user script, a structured shot triplet containing core elements is generated collaboratively by two large language models. Next, a storyboard is created and a keyframe sequence is generated using a camera-aware routing mechanism. Then, combining keyframes and camera parameters, independent video segments are generated through interpolation techniques. Finally, the segments are seamlessly spliced ​​together using trajectory-guided transition processing to obtain a complete long-form video. This solution, through phased refined control, optimized inter-frame transitions, and data support, not only solves the problem of insufficient fine-grained control in existing technologies but also improves the coherence of multi-shot sequences. Furthermore, it ensures generation quality without requiring overall training, meeting the needs of professional creative work.

[0022] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The core of this application is to provide a training-free method for generating cinematic-quality long videos, and a flowchart of one specific implementation is shown below. Figure 1 As shown, the method includes: S101. Obtain predefined control parameters and user scripts from the control parameter library. Based on the control parameters and user scripts, generate scripts through the collaborative work of two major language models to obtain structured shot triples.

[0024] Among them, predefined control parameters are core configuration information used to constrain the direction of video generation, including movie genre tags, number of shots, camera parameters, etc., providing clear guidance for generating videos that meet the requirements. User scripts are creative blueprints that carry the core plot and character interactions of the video, and are the basis for scene description generation. Dual language models refer to two language processing models with different functions but working together, respectively undertaking the tasks of scene construction and camera instruction conversion. Structured shot triplets are the core data carriers that integrate key video information, covering the initial shot, camera parameters, and ending shot, providing standardized input for subsequent storyboard production.

[0025] Optionally, step S101 may specifically include the following steps: S1011, The predefined control parameters include movie type tags, number of shots, and camera parameters; the dual language models include a first language model and a second language model.

[0026] Among them, the film genre tag is used to define the style attributes of the video, such as suspense, comedy, documentary, etc. Camera parameters are a set of parameters that control the shooting angle and motion state, including camera motion type, shooting focal length, and frame composition ratio. The first language model focuses on the structured description generation of scene content, while the second language model focuses on the visual conversion of camera commands. The two work together to achieve a precise mapping from creative needs to technical parameters.

[0027] S1012. Using the first large language model as a scene generator, based on the movie type label, number of shots and the core plot of the user script, generate a structured scene description including lighting, location, subject, action and crowd density.

[0028] The structured scene description is a collection of scene information organized according to fixed dimensions, covering key dimensions such as lighting, location, subject, action, and crowd density. Lighting can be categorized into bright, dim, and warm light types; location includes scenes such as city streets, forests, and indoor living rooms; subjects can be objects such as young women, pet dogs, and cars; actions include running, talking, and standing still; and crowd density is categorized into empty, sparse, and dense conditions, ensuring that scene information is comprehensive and standardized. This first major language model requires no additional training; leveraging its pre-trained scene understanding capabilities and learning of film and television production standards, it can directly generate standardized descriptions based on input information.

[0029] S1013. Using the second language model as a cinematographer, based on the abstract camera instructions corresponding to the camera parameters and combined with the structured scene description, the abstract camera instructions are converted into a precise visual description.

[0030] Abstract camera commands are concise camera operation instructions, such as "pull," "pan," and "overhead shot." Precise visual description translates these abstract commands into specific image requirements that can be recognized by the image generation model, specifying details such as focus, background treatment, and subject proportion. This second major language model also requires no additional training; it learns professional film and television shooting knowledge to establish a mapping relationship between abstract camera commands and visual effects, achieving precise command conversion.

[0031] S1014. Based on the structured scene description and the precise visual description, a structured shot triad is derived. The structured shot triad includes the initial frame, the camera parameters, and the final frame.

[0032] The initial frame describes the starting state of each shot, while the ending frame describes the final state of each shot after camera movement. Together with camera parameters, these two elements constitute the complete execution logic of the shot. The derivation process involves extracting core elements from the structured scene description and image requirements from the precise visual description, combining this with motion constraints from camera parameters to clarify the starting and ending states of the shot, thus forming standardized triplet data.

[0033] In this embodiment, the core input parameters and model division of labor are clarified in S1011, then a standardized scene description is generated in S1012, a precise visual description is converted in S1013, and finally a structured shot triplet is obtained in S1014, forming a complete script generation process.

[0034] For example, a cinematic long-form video with a healing rural travel theme is generated as a unified background: First, predefined control parameters are obtained from the control parameter library through S1011, clarifying that the movie type label is "healing", the number of shots is set to 3, and the camera parameters are "panning motion, focal length 85mm, and aspect ratio 4:3". At the same time, the division of labor between the two major language models is determined. The first major language model is the scene generation model, and the second major language model is the camera command conversion model. The core plot of the user script is set as "At 6 a.m., a 25-year-old female traveler comes to a mountain village, walks along the path by the stream, feels the breath of nature, and is accompanied by the sounds of birds and streams."

[0035] Next, S1012 is executed. The first language model acts as a scene generator, fully combining the established "healing" movie genre tag, the planning of 3 shots, and the core plot of "morning stroll along a rural stream" in the user script to accurately generate a structured scene description: "Lighting: Soft natural light in the early morning with a misty scattering effect, the light is warm and not dazzling; Location: A mountain village path along a stream, the road is paved with bluestone slabs, with green grass and scattered wildflowers on both sides, and scattered wooden houses in the distance; Subject: A 25-year-old female traveler, wearing a light-colored dress, carrying a canvas bag, and holding a sun hat; Action: Strolling slowly, occasionally stopping to look down at the wildflowers by the roadside, and occasionally raising her hand to brush the breeze across her cheek; Crowd density: Sparse, with an elderly person sitting quietly in front of a wooden house in the distance, and no other pedestrians."

[0036] Then, using the second language model of S1013 as a cinematographer, based on the abstract camera instruction of "panning motion" in the camera parameters, and closely combined with the newly generated structured scene description, the abstract camera operation instruction is transformed into a detailed and executable precise visual description: "The initial position of the camera is aimed at the panoramic view of the path by the stream, the picture includes the bluestone path, the grass and wildflowers on both sides and the wooden house in the distance, and the female traveler is located at the right third of the picture; the camera slowly pans horizontally, keeping the traveler always within the visible range of the picture during the panning, and the natural landscape in the background moves by at a constant speed, highlighting the openness and tranquility of the rural environment; finally, the camera pans to a close-up of the traveler's side, the traveler occupies 60% of the center of the picture, and the background is blurred to present green grass and morning mist, highlighting the sense of integration between the person and nature."

[0037] Finally, S1014 is executed to extract the core elements from the structured scene description, including the soft natural light environment in the early morning, the location of the mountain village stream path, the 25-year-old female traveler and related actions. At the same time, the initial state of the scene, the panning motion requirements, and the final close-up effect are extracted from the precise visual description. Combined with the motion constraints in the camera parameters "panning motion, focal length 85mm, and image composition ratio 4:3", the system derives the structured lens triplets: "Initial scene: Panoramic view of a bluestone path by a mountain village stream, with green grass and scattered wildflowers on both sides, and scattered wooden houses in the distance. The 25-year-old female traveler wearing a light-colored dress is located in the right third of the scene, with an image composition ratio of 4:3; Camera parameters: panning motion, focal length 85mm, and image composition ratio of 4:3; Ending scene: Close-up of the 25-year-old female traveler in profile, occupying 60% of the center of the scene, with a blurred green grass and morning mist in the background, and an image composition ratio of 4:3."

[0038] The above example is only one specific implementation of this application. In practical applications, the specific values ​​of the control parameters, the plot content of the user script, and the details of each descriptive dimension can be flexibly adjusted according to the specific needs of film and television creation. This application does not limit this.

[0039] This application, through the collaborative work of two major language models, accurately transforms user scripts and control parameters into standardized structured shot triples, achieving semantic-visual precision mapping of camera commands. This ensures comprehensive and standardized scene descriptions and clear and specific camera controls, effectively improving the relevance and accuracy of video generation.

[0040] S102. Based on the structured lens triplet, a storyboard is created using a camera-aware routing mechanism to generate a keyframe sequence.

[0041] Among them, the camera-aware routing mechanism is a technical mechanism that can adaptively match and process models based on camera motion type. Its core lies in the dynamic routing algorithm, which enables accurate model selection under different motion scenarios. Storyboard creation is the process of converting structured shot triplets into visual keyframes, a core step connecting the script and video clips. A keyframe sequence is a dataset formed by arranging multiple high-fidelity keyframes in chronological order, with each keyframe carrying core visual information for its corresponding moment.

[0042] Optionally, step S102 may specifically include the following steps: S1021. Based on the structured scene description in the structured lens tripartite and combined with the image composition requirements corresponding to the camera parameters, a high-fidelity starting image is generated.

[0043] Among them, a high-fidelity starting shot refers to a shot that has clear details, realistic color reproduction, and meets the requirements of scene description and composition, and can accurately present the core elements in the structured scene description.

[0044] S1022. Based on the high-fidelity starting image, and combined with the precise visual description and camera motion trend in the structured lens tripartite, generate continuous keyframes.

[0045] Among them, camera motion trend is the direction and pattern of image changes derived from the motion type in the camera parameters, such as panning and tracking. Continuous keyframes are a series of images generated at time intervals. Each frame inherits the visual elements of the previous frame and changes gradually according to precise visual description and camera motion trend to ensure visual continuity between frames.

[0046] S1023. Using the dynamic routing algorithm of the camera-aware routing mechanism, based on the camera motion type corresponding to the camera parameters in the structured lens triplet, select the optimal model to generate a high-fidelity keyframe sequence, wherein the static camera corresponds to the Gemini model and the panning camera corresponds to the Tongyi Qianwen model.

[0047] The dynamic routing algorithm is the core algorithm of the camera-aware routing mechanism. It is used to analyze the camera motion type and model adaptability, and output the optimal model selection result. The optimal model refers to the model that performs better in terms of image generation quality and coherence for a specific camera motion type. The Gemini model corresponds to static cameras, and the Tongyi Qianwen model corresponds to panning cameras. Neither the Gemini model nor the Tongyi Qianwen model requires additional training. Through pre-training, they have accumulated advantages in static image generation and dynamic panning scene image generation, respectively, and can be directly used by the algorithm.

[0048] In this embodiment, a high-fidelity starting frame is first generated based on the structured scene description and composition requirements in S1021. Then, continuous keyframes are generated based on the starting frame in S1022. Finally, the dynamic routing algorithm in S1023 selects the adaptation model and integrates them to obtain a high-fidelity keyframe sequence, forming a complete storyboard production process and providing standardized materials for the subsequent generation of cinematic long videos.

[0049] For example, let's take a scenario where a cinematic-quality long video is generated using a unified theme of healing rural travel as the shooting theme: First, execute S1021 to extract the structured scene description from the structured lens triplets. Specifically, the lighting is soft natural light in the early morning with a light mist scattering effect; the location is a mountain village stream path paved with bluestone slabs; the sides are green grass and scattered wildflowers; there are scattered wooden houses in the distance; the main subject is a 25-year-old female traveler wearing a light-colored dress and carrying a canvas bag; the action is a slow walk; and the crowd density is sparse. Combining the 4:3 aspect ratio and 85mm focal length image composition requirements in the camera parameters, a high-fidelity starting image is generated through the text-to-image model.

[0050] The scene is presented in a 4:3 cinematic aspect ratio, clearly showing the bluestone path by the stream, the green grass on both sides and the scattered wildflowers. In the distance, the wooden houses are shrouded in the morning mist. A 25-year-old female traveler wearing a light-colored dress and carrying a canvas bag is located in the right third of the frame. The overall lighting is warm and soft, and the details of the grass, wildflowers and wooden houses are highly reproduced, perfectly matching the theme style of a healing rural travel movie.

[0051] Next, based on the generated high-fidelity starting frame using S1022, and combining precise visual descriptions—namely, horizontal camera panning, ultimately presenting a close-up of the traveler's profile with a blurred background highlighting the harmony between the person and nature—and the camera movement trend—namely, a uniform horizontal panning at a speed matching the soothing rhythm of a healing video—continuous keyframes are generated through an image-to-image model. Frame 1 is the starting frame. Frames 2-5 gradually present the camera panning process, with the traveler's position in the frame slowly shifting to the left. The grass, wildflowers, and wooden houses in the background glide by at a uniform speed, resulting in a natural and smooth transition. Frame 6 presents a close-up of the traveler's profile, with a relaxed and natural facial expression. The blurred background reveals green grass and morning mist, emphasizing the healing atmosphere. These six keyframes are arranged in chronological order, fully presenting the core visual evolution of a single shot.

[0052] Finally, in step S1023, the dynamic routing algorithm is used to analyze the motion type corresponding to the camera parameters. This motion type is panning motion and belongs to the translation category. The adaptability determination process of the dynamic routing algorithm is implemented through the following logic: First, the features of the camera motion type are extracted, including translation, motion speed, and changes in the field of view. Then, an adaptability scoring system between the motion type features and the candidate model is established. The scoring adopts a weighted summation method, where the motion matching weight is 0.6, the quality performance weight is 0.4, and the model's matching degree with the motion type and the model's historical generation quality score in the corresponding scene are both in the range of 0-1.

[0053] For the panning motion in this example, the Tongyi Qianwen model has a motion matching score of 0.93 and a history generation quality score of 0.91, while the Gemini model has a motion matching score of 0.72 and a history generation quality score of 0.90. Weighted summation shows that the Tongyi Qianwen model has a higher fit score than the Gemini model. Because of the higher fit score, the Tongyi Qianwen model was chosen to optimize and integrate the generated 6 consecutive keyframes, ultimately resulting in a high-fidelity keyframe sequence that meets cinematic quality requirements and is suitable for the healing rural travel theme.

[0054] The above example is only one specific implementation of this application. In practical applications, the model selection logic, number of keyframes and image detail requirements can be flexibly adjusted according to the movie theme, camera movement type, scene complexity, etc. This application does not limit these aspects.

[0055] This application generates a sequence of keyframes that conforms to the film's shooting theme, scene requirements, and camera instructions by accurately generating high-fidelity starting frames, deriving continuous keyframes, and combining dynamic routing algorithms to select an adaptation model. This ensures that the keyframes are highly matched with the script's intent and the type of camera movement, effectively guaranteeing the overall quality of the generated cinematic long video.

[0056] S103. Based on the keyframe sequence, the camera parameters in the predefined control parameters, and the scene description, generate independent video segments using keyframe-to-video interpolation technology.

[0057] Keyframe-to-video interpolation is a core technology that transforms discrete keyframe sequences into complete video clips by generating consecutive intermediate frames between adjacent keyframes. It encompasses four key stages: spatiotemporal constraint coding, intelligent interpolation generation, geometric stability verification, and integrated output. Independent video clips are complete video materials corresponding to a single shot, possessing visual coherence and complete content, and can be directly used for splicing and integrating cinematic-quality long videos.

[0058] Optionally, step S103 may specifically include the following steps: S1031. Extract adjacent keyframe pairs from the keyframe sequence, combine them with the camera parameters, and convert the adjacent keyframe pairs and camera parameters into spatiotemporal control signals through encoding processing.

[0059] Among them, adjacent keyframe pairs are two keyframes that are sequentially adjacent in the keyframe sequence, carrying the core visual information of the image from its initial state to its changing state. Spatiotemporal control signals are constraint instructions that fuse spatial image features with temporal motion patterns, used to define the generation range, change trend, and motion rules of intermediate frames, ensuring that interpolation generation does not deviate from expectations. The encoding process employs a spatiotemporal constraint coding algorithm, which requires no additional training. It integrates the pixel features, composition information, and camera motion parameters of the keyframes into a standardized control signal.

[0060] S1032. Using the spatiotemporal control signal as a constraint, the image changes between adjacent keyframe pairs are deduced using the Hero AI interpolation algorithm to generate a continuous intermediate frame sequence.

[0061] Among them, the Hello AI interpolation algorithm is an intelligent interpolation algorithm based on deep learning. It requires no additional training and learns the natural inter-frame variation patterns of massive amounts of video through pre-training. Under the constraints of spatiotemporal control signals, it can accurately deduce the scene transition process between adjacent keyframes. The intermediate frame sequence is a series of continuous images between adjacent keyframes. Each frame inherits the visual features of the previous frame and gradually transitions to the state of the subsequent keyframes, ensuring smooth and seamless scene changes.

[0062] S1033. Perform geometric stability verification on the intermediate frame sequence. By detecting the pixel displacement between frames, the matching degree of the main body contour, and the consistency of background elements, automatically identify the inconsistency problem between frames and repair it accordingly. The inconsistency problem between frames includes main body drift and background abrupt change.

[0063] Geometric stability verification is a mechanism used to detect and repair inter-frame anomalies. It achieves image consistency verification through three core methods: pixel matching, contour comparison, and background tracking. Subject drift refers to the phenomenon where the position of the subject deviates from the expected trajectory in inter-frame images, while background abrupt change refers to the phenomenon where background elements change suddenly without a reasonable transition. This verification process requires no additional training. By setting preset similarity thresholds and trajectory deviation thresholds, it automatically identifies abnormal frames and performs pixel correction, contour alignment, and background smoothing to ensure visual consistency between frames.

[0064] S1034. Using keyframe-to-video interpolation technology, the repaired intermediate frame sequence is integrated to form a visually coherent and content-complete independent video segment.

[0065] The integration process involves seamlessly stitching the repaired intermediate frame sequence with the corresponding adjacent keyframes in chronological order to form a complete shot. Through unified scheduling of keyframe-to-video interpolation technology, it ensures that intermediate frames and keyframes maintain consistency in visual style, color tone, and motion trends, ultimately outputting a complete and naturally transitioning independent video clip that meets the quality requirements of cinematic shots.

[0066] In this embodiment, adjacent keyframe pairs are first extracted in S1031 and encoded into spatiotemporal control signals in combination with camera parameters. Then, an intermediate frame sequence is generated based on the signal in S1032. After detecting and repairing the inconsistency between frames in S1033, independent video segments are integrated in S1034 to form a complete video segment generation process. The generation quality can be guaranteed without training throughout the process.

[0067] In practical applications, an example is given of a scenario where a "healing rural travel" theme is used as the unified shooting theme, and cinematic-quality long videos are generated around this theme: First, adjacent keyframe pairs are extracted from the keyframe sequence using S1031, specifically frame 3 "The traveler strolls to the rocks by the stream and looks at the stream from the side" and frame 4 "The traveler bends down to approach the stream and observes the reflection in the water". Combined with the camera parameters such as panning motion and focal length 85mm, the pixel features, traveler position information and panning motion speed parameters of the two frames are extracted using a spatiotemporal constraint coding algorithm. This information is then integrated and converted into a spatiotemporal control signal to clarify the range of image changes, motion speed and traveler position constraint boundaries of the intermediate frame.

[0068] Secondly, using the generated spatiotemporal control signal as a constraint, the S1032 algorithm, employing the Hero AI interpolation algorithm, deduces the scene changes between two frames, generating a sequence of five consecutive intermediate frames. The first intermediate frame shows the traveler halfway towards the rocks by the stream, gradually shifting their body posture towards the water. The second to fourth intermediate frames gradually show the traveler bending over, with the camera maintaining a synchronized panning motion, resulting in a smooth transition of perspective. The fifth intermediate frame shows the traveler approaching the stream, seamlessly connecting with subsequent keyframes. The overall motion trend of the intermediate frame sequence fully conforms to the constraints of the spatiotemporal control signal.

[0069] Then, step S1033 is executed to perform geometric stability verification on the generated intermediate frame sequence. By detecting the pixel displacement of the traveler between frames, a slight drift in the traveler's head position was found in the third intermediate frame, deviating from the expected trajectory. By comparing the subject contour matching degree, it was confirmed that there was no contour deformation issue. By tracking the grass, stream, and distant wooden house in the background, no sudden changes in the background were found. To address the head drift issue, the system automatically performed pixel correction, adjusting the traveler's head position to the expected trajectory, completing the anomaly repair, and ensuring the consistency of the images between frames.

[0070] Finally, S1034 utilizes keyframe-to-video interpolation technology to integrate and splice the five repaired intermediate frames with their corresponding adjacent keyframes in chronological order, forming a 3-second independent video clip. In this clip, the traveler's strolling and bending movements are fluid and natural, the camera panning is smooth and stable, the background elements remain consistent, and the overall visual style matches the film's theme of a healing rural travel experience, fully meeting the quality requirements of cinematic shots.

[0071] The above example is only one specific implementation of this application. In practical applications, the number of intermediate frames, verification threshold and image detail requirements can be flexibly adjusted according to keyframe interval, scene complexity, movie theme style, etc. This application does not limit these aspects.

[0072] In another specific implementation, if a background abruptness occurs in the intermediate frame sequence, such as the sudden appearance of urban building elements unrelated to the healing rural scene, the geometric stability verification will automatically identify the abnormal frame. Through background replacement and smooth transition processing, the abrupt background will be replaced with grass or stream elements that conform to the logic of the rural scene, ensuring visual coherence between frames and not destroying the overall theme style of the film.

[0073] This application generates visually coherent and complete independent video clips through spatiotemporal control signal constraints, intelligent interpolation generation, geometric stability verification, and integrated processing. It effectively avoids problems such as subject drift and sudden background changes, ensuring that the clips meet camera instruction requirements and film theme style, and providing high-quality and highly adaptable shot materials.

[0074] S104. Based on the independent video segments, a transition segment is generated through trajectory-guided seamless transition processing, and the independent video segments and the transition segment are spliced ​​together using first-order continuous interpolation technology to obtain a cinematic long video.

[0075] The trajectory-guided seamless transition processing is a core technology that establishes motion correlations between segments, fits smooth trajectories, and generates transition frames to achieve natural connections between adjacent video segments. It encompasses four stages: feature point matching, static frame selection, trajectory fitting, and transition generation. A transition segment is an intermediate sequence of frames that connects two adjacent independent video segments, used to eliminate visual gaps between them. First-order continuous interpolation technology ensures a consistent motion trend in the spliced ​​images. By aligning image features frame by frame, it achieves seamless integration of independent video segments and transition segments.

[0076] Optionally, step S104 may specifically include the following steps: S1041. Using a third-generation collaborative tracker, feature point matching is performed on the frames of adjacent independent video segments to establish sparse corresponding points between adjacent frames. The accuracy of the sparse corresponding points is controlled within ±0.5 pixels.

[0077] Among them, the third-generation collaborative tracker is a visual tracking tool with high-precision feature point matching capabilities. It requires no additional training and learns the logic for extracting and matching image feature points through pre-training, enabling it to quickly identify common features in adjacent frames. Sparse corresponding points are key feature points in adjacent frames that are related in position, such as key points of the main body contour and feature points of fixed background elements, with an accuracy controlled within ±0.5 pixels.

[0078] S1042. Adaptive window truncation processing is applied to the frame sequences of adjacent independent video segments to dynamically identify and delete static frames with a motion threshold of less than 0.1 pixels / frame.

[0079] Adaptive window truncation is a technique for dynamically filtering frame sequences, analyzing inter-frame motion changes in real time through a sliding window. A motion threshold is used to determine the dynamic attributes of a frame; frames with a threshold of less than 0.1 pixels per frame are considered static frames. These frames lack effective motion information, and deleting them reduces interference with the smoothness of transitions, ensuring that transition segments focus on the effective changes in the image.

[0080] S1043. Based on the sparse corresponding points, determine the correlation between the positions of the frames. Using parameters with values ​​ranging from 0 to 1 as variables, and combining the initial position and initial velocity of the independent video segment with the termination position and termination velocity of the adjacent segments, construct a cubic Hermit spline path and perform trajectory fitting to obtain a smooth trajectory.

[0081] The cubic Hermitian spline path is a smooth path model that balances position and velocity constraints, suitable for describing transitional motion trajectories between segments. The parameter t ranges from 0 to 1, with t=0 corresponding to the termination state of the previous independent video segment and t=1 corresponding to the starting state of the next independent video segment. Trajectory fitting is based on the positional correlation of sparse corresponding points, combining four key parameters: initial position, initial velocity, termination position, and termination velocity. The path model calculates a continuous and smooth transition trajectory, ensuring a consistent and natural motion trend.

[0082] S1044. Using the smooth trajectory as a guide, generate a transition segment that connects adjacent independent video segments.

[0083] The generation of transition segments is achieved through a frame generation model. This model does not require additional training. It relies on pre-trained image generation capabilities, uses smooth trajectories as motion constraints, and refers to the visual style, color tone, and content elements of adjacent independent video segments to generate a sequence of intermediate transitional images, thus achieving a natural connection between the two segments in terms of content and motion.

[0084] S1045. Using first-order continuous interpolation technology, the independent video segments and the corresponding transition segments are spliced ​​frame by frame to form a coherent and complete cinematic long video.

[0085] Among them, first-order continuous interpolation technology achieves seamless splicing between segments by ensuring the continuity of adjacent frames in terms of position and motion speed. During the frame-by-frame splicing process, this technology aligns the ending frame of the previous segment, all frames of the transition segment, and the starting frame of the next segment, eliminating visual jumps between frames and ultimately integrating all segments to form a complete cinematic long video.

[0086] In this embodiment, sparse corresponding points of adjacent segments are first established in S1042, static frames are deleted in S1043, a smooth trajectory is fitted in S1044, transition segments are generated in S1045, and finally, frames are stitched together in S1046 to form a complete long video generation process. The continuity of multiple shots can be guaranteed without training throughout the process.

[0087] For example, consider a scenario where a cinematic-quality long-form video is generated around a unified theme of healing rural travel: First, two adjacent independent video clips are selected using S1041. The first clip is "Traveler strolling along the stream and observing the reflection," and the second clip is "Traveler walking along the path towards the cabin." The first and last frames of the two clips are matched with feature points using a third-generation collaborative tracker. Sparse corresponding points are established on the traveler's shoulders, head outline, and fixed locations in the background such as rocks along the stream and corners of the cabin. The accuracy of all corresponding points is controlled within ±0.5 pixels, providing a precise benchmark for inter-frame position association.

[0088] Secondly, S1042 performs adaptive window truncation processing on the frame sequences of two adjacent independent video segments. The sliding window analyzes the motion changes between frames one by one and dynamically identifies static frames with a motion threshold of less than 0.1 pixels / frame—including the two frames at the end of the first segment where the traveler is still observing the reflection and the one frame at the beginning of the second segment where the cabin is in the distance. These static frames are automatically deleted, and the frame sequence with effective motion information is retained.

[0089] Then, S1043 is executed to determine the inter-frame positional relationship based on the established sparse corresponding points. Using the parameter t, which ranges from 0 to 1, as a variable, the initial position of the first segment (coordinates of the traveler beside the rocks by the stream) and the initial speed (movement speed while walking) are extracted, as well as the ending position of the second segment (coordinates of the traveler in front of the cabin) and the ending speed (deceleration speed when approaching the cabin) are extracted. These are then substituted into a cubic Hermit spline path model for trajectory fitting to obtain a smooth motion trajectory from the stream to the cabin. The trajectory conforms to the direction of the country road, and the movement speed gradually becomes stable, which is consistent with the healing style.

[0090] Next, guided by the fitted smooth trajectory using S1044, a transition segment is generated through a frame generation model. The transition segment consists of 8 frames, sequentially showing the traveler getting up from the stream and slowly walking along the path to the cabin. The visual elements in the frame, such as grass, wildflowers, and morning mist, are consistent with the two independent video segments before and after, and the motion trajectory completely follows the fitted path, achieving a natural transition between content and motion.

[0091] Finally, step S1045 uses first-order continuous interpolation to stitch the first independent video clip, the transition clip, and the second independent video clip frame by frame. The frame-by-frame alignment of the visual features and motion speed of each clip ensures no visual jumps between the end frame of the first clip and the start frame of the transition clip, and between the end frame of the transition clip and the start frame of the second clip. This results in a 12-second, visually coherent, and complete healing-themed rural travel movie-quality long video with three camera transitions.

[0092] The above example is only one specific implementation of this application. In practical applications, the number of sparse corresponding points, the static frame judgment threshold and the length of the transition segment can be flexibly adjusted according to the movie theme, the number of segments, the requirements for shot switching, etc. This application does not limit these aspects.

[0093] In another specific implementation, if the theme of the film is "urban dynamics documentary", and adjacent independent video clips involve rapid shot transitions, such as the connection between street traffic and building panoramas, the third-generation collaborative tracker will prioritize matching dynamic feature points in the scene, adaptive window truncation processing will increase the static frame judgment threshold, trajectory fitting will generate a smoother trajectory that fits the fast movement, and the number of frames in the transition clips will be appropriately reduced to ensure that the spliced ​​long video conforms to the fast-paced style of urban documentary while maintaining visual coherence.

[0094] This application generates naturally connected transition segments through trajectory-guided seamless transition processing, and achieves precise splicing by combining first-order continuous interpolation technology. This effectively solves the continuity problem of multi-camera videos, ensuring smooth and complete cinematic long videos that match the preset movie theme style.

[0095] Optionally, it further includes: pre-constructing a balanced high-quality video dataset of 5,000 videos before generating the cinematic long video; the method described in claim 1 adopts a four-stage decoupling architecture, which corresponds to script generation action, storyboard production action, video clip generation action, and seamless transition and splicing action, respectively. The components corresponding to each stage action can be upgraded independently without retraining the entire system; the cinematic long video supports multi-dimensional joint control of 17 camera movements and 15 film types, with an average duration of 12 seconds, and supports 3-camera switching.

[0096] Among them, the balanced high-quality video dataset is a standardized video collection that covers a variety of film types and camera motion modes, is evenly distributed, has clear images, and is fully annotated, providing data support for the entire method process; the four-stage decoupled architecture breaks down the generation process into four independent but related stages: script generation, storyboard production, video clip generation, and seamless transition and splicing. Each stage component operates independently and has standardized interfaces, supporting independent component upgrades without retraining the entire system; multi-dimensional joint control means that long videos can simultaneously respond to two types of control requirements: camera motion and film type, achieving personalized creative adaptation.

[0097] In this embodiment, a balanced high-quality video dataset is pre-built to support subsequent steps, a four-stage decoupling architecture is adopted to ensure system flexibility, and multi-dimensional control functions and specification parameters for long videos are configured to form basic guarantees and functional configurations.

[0098] For example, consider a scenario where a cinematic-quality long-form video is generated around a unified theme of healing rural travel: First, a balanced, high-quality video dataset of 5,000 videos was pre-constructed. The dataset covers 15 film genres and 17 camera motion modes, with even distribution across all dimensions. The videos are clear and have complete annotation information. Among them, videos related to healing rural travel scenes are annotated with key information such as natural morning light, slow panning motion, and green grass background, which directly provides accurate data reference for subsequent script generation, keyframe generation, and other processes.

[0099] Secondly, a four-stage decoupled architecture is adopted to build the system. The four stages correspond to dedicated functional components for script generation, storyboard production, video clip generation, and seamless transition and splicing. Information is exchanged between components through a standardized data format. If the keyframe generation accuracy of storyboard production needs to be optimized, only the corresponding component needs to be upgraded. There is no need to modify other components or retrain the entire system. After the upgrade, the component can still be seamlessly compatible with the original architecture.

[0100] Finally, configure the long video function and specifications. Users select the healing film type and panning camera movement, and the system supports joint control of this dual selection and automatically matches the generation logic; at the same time, according to the specifications of an average duration of 12 seconds and support for 3-camera switching, the three independent video clips "Traveler strolling by the stream", "Traveler walking towards the wooden house" and "Traveler sitting quietly in front of the wooden house" are connected by transition clips to generate a complete healing rural travel long video to meet creative needs.

[0101] The above example is only one specific implementation of this application. In practical applications, the content of the dataset, the details of the architecture components and the long video specifications can be flexibly adjusted according to the dataset coverage scenario, system upgrade requirements, user creation requirements, etc. This application does not limit these aspects.

[0102] In another specific implementation, if a user needs to generate a long urban detective video of the suspense type, they can choose the joint control of the suspense movie type and the tracking camera movement. Based on the relevant video data in the dataset, the system uses a four-stage decoupled architecture to work together to generate a suspense-style long video with a duration of 12 seconds that supports 3-lens switching. The system can still maintain stable operation when components are upgraded.

[0103] This application provides stable support by constructing a balanced, high-quality video dataset, adopts a four-stage decoupling architecture to enhance system flexibility and scalability, and configures multi-dimensional joint control and adaptability specifications to provide comprehensive basic guarantees for the generation of cinematic long videos, meet the diverse creative needs of users, and ensure that the generation process is efficient and flexible and the results are highly adaptable.

[0104] Optionally, each stage of the method supports manual intervention and conforms to film and television production standards; the camera-aware routing mechanism is a model routing mechanism for camera motion type perception; the application process of the keyframe-to-video interpolation technology includes the generation of dual-condition constraints of keyframes and camera commands; the trajectory-guided seamless transition processing includes trajectory-guided smooth spatiotemporal interpolation.

[0105] Among them, the camera motion type-aware model routing mechanism is a technical mechanism that can automatically identify the camera motion type and match the appropriate model to ensure the quality of image generation under different motion scenarios; the dual-condition constraint generation of keyframes and camera commands refers to using the image features of keyframes and the motion requirements of camera commands as constraints when generating video clips to ensure that the generated content accurately matches expectations; the trajectory-guided smooth spatiotemporal interpolation is a technology that generates transitional images in the time and space dimensions based on fitted smooth trajectories to ensure natural transitions between clips.

[0106] In one specific implementation, each stage of the method supports manual intervention. For example, the structured scene description can be manually adjusted during the script generation stage, and image details can be modified after keyframe generation. All intervention operations conform to the standard procedures of professional film and television production. The camera-aware routing mechanism first identifies the camera motion type and then matches the corresponding optimal model, without requiring manual specification. When applied, the keyframe-to-video interpolation technology simultaneously references the image information of the keyframes and the camera's motion commands, with dual constraints ensuring that the generated video clips do not deviate from the requirements. The trajectory-guided seamless transition processing is based on a smooth trajectory, generating images that transition naturally in the spatiotemporal dimension between adjacent clips, eliminating splicing gaps.

[0107] like Figure 2As shown, the end-to-end execution flow of the training-free cinematic long video generation method of this application is as follows: taking the predefined parameters of the control parameter library and the user script as input, the two first enter the script generation module, and output a structured shot triplet with the support of dual LLM collaborative work; the triplet is sent to the storyboard production module on the one hand, and triggers the camera-aware routing mechanism on the other hand to jointly generate a keyframe sequence; subsequently, the keyframe sequence enters the video segment generation module, and generates independent video segments with the support of FLF2V interpolation technology; the independent video segments are simultaneously sent to the seamless transition module and the C1 continuous interpolation stage to generate transition segments for connection, and finally the long video synthesizer integrates the independent segments and transition segments to output a cinematic long video, and simultaneously generates metadata containing type tags, camera parameters, and scene descriptions.

[0108] Figure 3 This is a schematic diagram illustrating a specific implementation of a training-free cinematic long video generation system provided in this application. (Refer to...) Figure 3 The system may include: The script generation module 31 is used to obtain predefined control parameters and user scripts from the control parameter library, and generate scripts by working collaboratively with two major language models based on the control parameters and user scripts to obtain structured shot triples. Storyboard creation module 32 is used to create a storyboard based on the structured shot triplet and through a camera-aware routing mechanism, generating a keyframe sequence. The video clip generation module 33 is used to generate independent video clips based on the keyframe sequence, camera parameters in the predefined control parameters, and scene description, using keyframe-to-video interpolation technology. The seamless transition module 34 is used to generate a transition segment based on the independent video segment through trajectory-guided seamless transition processing, and to splice the independent video segment and the transition segment using first-order continuous interpolation technology to obtain a cinematic long video.

[0109] The training-free cinematic long video generation system of this application embodiment is used to implement the aforementioned training-free cinematic long video generation method. Therefore, the specific implementation of the training-free cinematic long video generation system can be found in the embodiment section of the training-free cinematic long video generation method above. The specific implementation can be referred to the description of the corresponding embodiment, which will not be repeated here.

[0110] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of any of the above-described training-free cinematic long video generation methods.

[0111] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of any of the above-described training-free cinematic long video generation methods.

[0112] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0113] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the method for generating cinematic long videos without training.

[0114] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0115] The foregoing has provided a detailed description of a training-free method and system for generating cinematic-quality long videos. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for generating cinematic-quality long videos without training, characterized in that, include: Predefined control parameters and user scripts are obtained from the control parameter library. Based on the control parameters and user scripts, scripts are generated through the collaborative work of two major language models to obtain structured shot triples. Based on the structured lens triplet, a storyboard is created using a camera-aware routing mechanism to generate a keyframe sequence. Based on the keyframe sequence, the camera parameters in the predefined control parameters, and the scene description, independent video segments are generated using keyframe-to-video interpolation technology. Based on the independent video segments, a transition segment is generated through trajectory-guided seamless transition processing. Then, using first-order continuous interpolation technology, the independent video segments and the transition segment are spliced ​​together to obtain a cinematic long video.

2. The method according to claim 1, characterized in that, Based on the keyframe sequence, the camera parameters in the predefined control parameters, and the scene description, independent video segments are generated using keyframe-to-video interpolation technology, including: Extract adjacent keyframe pairs from the keyframe sequence, combine them with the camera parameters, and convert the adjacent keyframe pairs and camera parameters into spatiotemporal control signals through encoding processing; Using the spatiotemporal control signal as a constraint, the image changes between adjacent keyframe pairs are deduced through the Hero AI interpolation algorithm to generate a continuous intermediate frame sequence. Geometric stability verification is performed on the intermediate frame sequence. By detecting the pixel displacement between frames, the matching degree of the main body contour, and the consistency of background elements, inconsistencies between frames are automatically identified and repaired in a targeted manner. The inconsistencies between frames include main body drift and background abrupt changes. By using keyframe-to-video interpolation technology, the repaired intermediate frame sequence is integrated to form a visually coherent and content-complete independent video segment.

3. The method according to claim 1, characterized in that, Based on the control parameters and the user script, a script is generated through the collaborative work of two major language models, resulting in a structured shot triplet, including: The predefined control parameters include movie type tags, number of shots, and camera parameters; the dual language models include a first language model and a second language model. Using the first large language model as a scene generator, a structured scene description including lighting, location, subject, action, and crowd density is generated based on the movie type label, number of shots, and core plot of the user script. Using the second language model as a cinematographer, based on the abstract camera instructions corresponding to the camera parameters and combined with the structured scene description, the abstract camera instructions are converted into precise visual descriptions; Based on the structured scene description and the precise visual description, a structured shot triad is derived, which includes the initial shot, the camera parameters, and the final shot.

4. The method according to claim 1, characterized in that, Based on the structured shot triplet, a storyboard is created using a camera-aware routing mechanism, generating a keyframe sequence, including: Based on the structured scene description in the structured lens tripartite and combined with the image composition requirements corresponding to the camera parameters, a high-fidelity starting image is generated. Based on the high-fidelity starting frame, and combined with the precise visual description and camera motion trend in the structured lens triplet, continuous keyframes are generated. The dynamic routing algorithm using the camera-aware routing mechanism selects the optimal model based on the camera motion type corresponding to the camera parameters in the structured lens triplet to generate a high-fidelity keyframe sequence, where the static camera corresponds to the Gemini model and the panning camera corresponds to the Tongyi Qianwen model.

5. The method according to claim 1, characterized in that, Based on the independent video segments, a transition segment is generated through trajectory-guided seamless transition processing. Then, using first-order continuous interpolation, the independent video segments and the transition segment are stitched together to obtain a cinematic long video, including: The third-generation collaborative tracker performs feature point matching on the frames of adjacent independent video segments, and establishes sparse corresponding points between adjacent frames. The accuracy of the sparse corresponding points is controlled within ±0.5 pixels. Adaptive window truncation is applied to the frame sequences of adjacent independent video segments to dynamically identify and delete static frames with a motion threshold of less than 0.1 pixels / frame. Based on the sparse corresponding points, the correlation between the positions of the frames is determined. Using parameters with values ​​ranging from 0 to 1 as variables, and combining the initial position and initial velocity of the independent video segment with the termination position and termination velocity of the adjacent segments, a cubic Hermit spline path is constructed, and a smooth trajectory is obtained by trajectory fitting. Guided by the smooth trajectory, a transition segment connecting adjacent independent video segments is generated; The independent video segments and the corresponding transition segments are spliced ​​frame by frame using first-order continuous interpolation technology to form a coherent and complete cinematic long video.

6. The method according to claim 1, characterized in that, Also includes: Before generating the cinematic-quality long video, a balanced, high-quality video dataset of 5,000 videos is pre-built; The method described in claim 1 adopts a four-stage decoupling architecture, which corresponds to script generation action, storyboard production action, video clip generation action and seamless transition and splicing action respectively. The components corresponding to each stage action can be upgraded independently without retraining the entire system. The cinematic long video supports multi-dimensional joint control of 17 camera movements and 15 film genres, with an average duration of 12 seconds and supports 3-lens switching.

7. The method according to claim 1, characterized in that, The method supports manual intervention in each stage of the process and complies with film and television production standards; the camera perception routing mechanism is a model routing mechanism for camera motion type perception; the application process of the keyframe to video interpolation technology includes the generation of dual-condition constraints of keyframes and camera commands; the trajectory-guided seamless transition processing includes trajectory-guided smooth spatiotemporal interpolation.

8. A training-free cinematic long video generation system, characterized in that, include: The script generation module is used to obtain predefined control parameters and user scripts from the control parameter library, and generate scripts based on the control parameters and user scripts through the collaborative work of two major language models, resulting in structured shot triples. The storyboard creation module is used to create a storyboard based on the structured shot triplet and through a camera-aware routing mechanism, generating a keyframe sequence. The video clip generation module is used to generate independent video clips based on the keyframe sequence, camera parameters in the predefined control parameters, and scene description, using keyframe-to-video interpolation technology. A seamless transition module is used to generate a transition segment based on the independent video segment through trajectory-guided seamless transition processing, and to splice the independent video segment and the transition segment using first-order continuous interpolation technology to obtain a cinematic long video.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the training-free cinematic long video generation method as described in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the training-free cinematic long video generation method as described in any one of claims 1 to 7.

Images