AI video content integrated generation method and system based on script structuring

Abstract

The invention discloses an AI video content integrated generation method and system based on script structuring, and relates to the technical field of artificial intelligence video generation. The method comprises the following steps: receiving an unstructured natural language story text, and disassembling the unstructured natural language story text into structured JSON data including a mirror number, a picture description, a role list and a scene list; packaging the AI generation request into an asynchronous task and distributing the asynchronous task; and executing a generation operation based on the structured JSON data and role vision setting, and returning a result. The system comprises a user interaction module, a business logic processing module, a task scheduling module and an intelligent generation service module which are connected through a protocol to form a data closed loop. The technical problems of data link splitting, poor content consistency and high modification cost of the existing video generation technology are solved, the full-link integrated creation from the script to the video is realized, the generation efficiency and the content quality are improved, and the modification and cooperation cost is reduced.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence video generation technology, specifically to an integrated method and system for generating AI video content based on script structuring. Background Technology

[0002] With the rapid development of artificial intelligence technology, AI-generated content (AIGC) is being used more and more widely in the field of video production. At present, video content production mostly adopts a discrete, multi-tool serial workflow, which usually involves three independent modules: script editing software, image generation tools (or material libraries), and non-linear video editing software.

[0003] The existing technology has the following significant drawbacks: First, the data link is fragmented, and there is a lack of automatic flow of contextual information between each stage. Users need to frequently switch between different software and manually copy and paste scripts, prompts, and other data. This is not only cumbersome but also prone to interruption of creation due to data transmission errors, which seriously reduces production efficiency. Second, the content consistency is poor. The image generation tool is independent of the script environment and cannot perceive the complete script context. This results in abrupt changes in the facial features and clothing details of the same character in different storyboards, or a disconnect between the scene style and the script description, making it difficult to meet the visual requirements of continuous narrative. Third, the modification cost is high. When there are minor modifications to the upstream script, the downstream generated storyboard images and video clips cannot be updated in a coordinated manner and need to be regenerated frame by frame, which greatly increases the creation time and resource consumption.

[0004] To address the aforementioned issues, the industry urgently needs an integrated generation solution that can break down module barriers and achieve data closure, in order to ensure the consistency of generated content and reduce creation and modification costs. Summary of the Invention

[0005] The purpose of this invention is to provide an AI video content integrated generation method and system based on script structuring, which realizes the integrated generation of the entire link from script text to video content, solves the technical problems of fragmented data links, poor content consistency, and high modification costs, improves generation efficiency, ensures content quality, and reduces modification costs.

[0006] The above-mentioned technical objective of the present invention is achieved through the following technical solution: A script-based AI video content integrated generation method includes the following steps: S1: Receives story text in unstructured natural language text format from the user through the user interaction module; S2: The business logic processing module receives the story text, calls the preset structured prompt word template, and drives the large language model to automatically decompose the story text into structured JSON data containing shot numbers, scene descriptions, character lists, and scene lists; S3: The task scheduling module encapsulates the AI ​​generation request corresponding to the structured JSON data into an asynchronous task and distributes it to the intelligent generation service module according to the task type; S4: The intelligent generation service module performs text, image, or video generation operations based on the structured JSON data and the associated character visual settings to obtain the generation results; S5: The task scheduling module receives the generated result, updates the task status, and sends it back to the business logic processing module, which then synchronizes it to the front-end interactive interface.

[0007] In a preferred embodiment, in step S2, the structured prompt template includes structural constraints, content constraints, and association constraints. The structural constraints require that the output JSON data must include four fields: a shot array, a role array, a scene array, and a props array. The content constraints require that the script field of each shot in the shot array includes shot type, camera movement, scene description, action, and dialogue, and that the role names use full names and prohibit the use of pronouns. The association constraints require that the role names referenced by the role field in the shot array must be names already defined in the role array.

[0008] In a preferred embodiment, in step S2, after obtaining the JSON data returned by the large language model, the business logic processing module compares the newly generated role and scene names with the existing entities in the current project in the database through an entity matching algorithm. If the names match exactly or there is a variant match, the ID of the existing entity is reused to achieve consistent association of cross-series role images.

[0009] In a preferred embodiment, the entity matching algorithm includes exact name matching and variant recognition. The variant matching is the matching of character names with attribute limitations, specifically including the matching forms of "full character name + age" and "full character name + identity".

[0010] In a preferred embodiment, step S3, the encapsulation and distribution process of the asynchronous task specifically includes: S31: The business logic processing module verifies the validity of the parameters of the AI-generated request. The parameters include the project ID and the target scene ID. After the verification is successful, an AiTask task entity is created in the database. The initial state of the AiTask task entity is set to pending. S32: The task producer of the task scheduling module pushes the AiTask task entity into the Redis message queue, the task consumer listens to the Redis message queue and obtains the task, and updates the status of the AiTask task entity to running. S33: The task consumer distributes the AiTask task entity to the processing unit corresponding to the intelligent generation service module according to the task type. The task type includes the storyboard image generation task shotImageGenerate and the video generation task videoGenerate.

[0011] In a preferred embodiment, in step S3, the task scheduling module sets differentiated timeout thresholds for different types of asynchronous tasks, wherein the timeout threshold for video generation tasks is set to 15 minutes and the timeout threshold for ordinary tasks is set to 5 minutes; if a task is not completed within the specified timeout threshold, the monitoring service forcibly sets the status of the task to failure and releases system resources.

[0012] In a preferred embodiment, in step S4, when the intelligent generation service module performs the image generation operation, it receives the character reference image or character feature LoRA model passed in by the business logic processing module, and combines it with the prompt words of the current storyboard to perform fusion generation, so as to ensure the visual consistency of the same character in different storyboard scenes.

[0013] In a preferred embodiment, in step S2, when generating structured JSON data, the business logic processing module synchronously parses the script dialogue and action descriptions to obtain the duration of the character's action corresponding to each scene, and embeds the duration of the character's action as a hidden field into the character list field of the structured JSON data; in step S4, when the intelligent generation service module calls the AI ​​generation model to generate character action scenes, it extracts the hidden duration of the character's action field and reuses it as a frame interval control parameter of the AI ​​generation model to regulate the frame interval of the output scene of the AI ​​generation model.

[0014] An AI video content integrated generation system based on script structure is used to implement the aforementioned AI video content integrated generation method based on script structure. The system includes a user interaction module, a business logic processing module, a task scheduling module, and an intelligent generation service module. Each module establishes a connection through a preset protocol to form a data closed loop. The user interaction module runs on the browser side, providing a script editing interface, a storyboard list view, and a multimedia editor. It integrates a status management unit to maintain the context data of the current project, episode, and storyboard. The business logic processing module runs in a Node.js environment, integrates the PrismaORM database adapter, and includes a storyboard service unit. It is used to receive story text input by the user, call the large language model through structured prompt word templates to generate structured JSON data, and perform entity matching operations. The task scheduling module is built on Redis message queue BullMQ and includes task producers and task consumers, which are used to encapsulate asynchronous AI task generation, task distribution and task status updates. The intelligent generation service module runs in the Python FastAPI environment. As a model gateway, it encapsulates the calling interfaces of different AI models to perform text, image and video generation operations and achieve visual consistency generation of characters.

[0015] In a preferred embodiment, the user interaction module and the business logic processing module establish a connection via the HTTP / WebSocket protocol for sending creation instructions and receiving generation results in real time; the business logic processing module is connected to the task scheduling module for transmitting AI generation requests; the task scheduling module is connected to the intelligent generation service module for driving the underlying AI model to perform specific generation operations; the intelligent generation service module uploads the generation results to the object storage OSS, and then sends them back to the user interaction module sequentially through the task scheduling module and the business logic processing module.

[0016] Compared with the prior art, the present invention has the following beneficial effects: 1. Enables integrated creation across the entire process, breaking down the modular barriers between script editing, image generation, and video editing. Data flows automatically at each stage, eliminating the need for users to manually switch software or copy data, significantly improving video generation efficiency; 2. Ensure consistency of generated content by using entity matching algorithms to achieve unified binding of character images across episodes, and combine this with character visual settings to avoid abrupt changes in character characteristics and scene style, thus meeting the visual requirements of continuous narrative; 3. Reduce modification and collaboration costs: Upstream script modifications can be linked to downstream storyboard data updates in real time, eliminating the need to regenerate frame by frame. At the same time, structured data facilitates multi-party collaboration and improves creative synergy. 4. Improve system stability and resource utilization by adopting a differentiated asynchronous task scheduling mechanism to adapt to the time consumption characteristics of different types of generated tasks, avoid resource blocking and connection timeouts, and ensure efficient system operation. Attached Figure Description

[0017] Figure 1 This invention relates to a flowchart of an AI video content integrated generation method based on script structuring. Detailed Implementation

[0018] The present invention will be further described in detail below with reference to the accompanying drawings.

[0019] This specific embodiment is merely an explanation of the present invention and is not intended to limit the invention. After reading this specification, those skilled in the art can make modifications to this embodiment without contributing any inventive step, but such modifications are protected by patent law as long as they are within the scope of the claims of the present invention.

[0020] Example 1 like Figure 1 As shown, an AI-powered integrated video content generation method based on script structuring includes the following steps: S1: Receives story text in unstructured natural language text format from the user through the user interaction module; S2: The business logic processing module receives the story text, calls the preset structured prompt word template, and drives the large language model to automatically decompose the story text into structured JSON data containing shot numbers, scene descriptions, character lists, and scene lists; S3: The task scheduling module encapsulates the AI ​​generation request corresponding to the structured JSON data into an asynchronous task and distributes it to the intelligent generation service module according to the task type; S4: The intelligent generation service module performs text, image, or video generation operations based on the structured JSON data and the associated character visual settings to obtain the generation results; S5: The task scheduling module receives the generated result, updates the task status, and sends it back to the business logic processing module, which then synchronizes it to the front-end interactive interface.

[0021] During execution, after the user inputs unstructured story text through the user interaction module on the browser (compatible with Chrome 100.0 and above, Edge 100.0 and above), the business logic processing module (running in Node.js 16.x and above) immediately receives the text and calls the preset structured prompt word template to drive the large language model (such as GeminiPro, GPT-4, with inference parameters set to temperature 0.7 and maximum generation length of 2048 tokens) to complete the decomposition of text into structured JSON data. The decomposed data format strictly follows the preset specifications to ensure that core information such as shot number and scene description are not missing. Subsequently, the task scheduling module (built on Redis 6.2.x and above, BullMQ 4.x and above) encapsulates the corresponding AI generation request into an asynchronous task and distributes it to the intelligent generation service module (running on Python 3.9.x and above, FastAPI 0.100.0 and above) according to the task type (image generation or video generation). The intelligent generation service module combines structured JSON data and associated character visual settings (character reference image resolution not less than 1024×1024 pixels, LoRA model weights set to 0.6-0.8) to perform the generation operation. The generated results (image format PNG / JPG, video format MP4, resolution supports 1080P / 4K) are fed back to the task scheduling module. The task scheduling module updates the task status (e.g., from running to completed) and sends it back to the business logic processing module. Finally, the business logic processing module synchronizes the generated results and task status to the front-end interactive interface via the WebSocket protocol for users to view, edit or perform further operations. This process achieves fully automated linkage from story text input to result feedback, breaking the fragmented status quo of script editing, image generation, and video generation in existing technologies. It eliminates the need for users to manually switch between different tools and transfer data. At the same time, through the collaborative cooperation of various modules, it lays the foundation for ensuring content consistency and reducing modification costs. Those skilled in the art can fully reproduce the entire AI video integrated generation process according to the clearly defined operating environment, parameter configuration, and process.

[0022] Furthermore, in step S2, the structured prompt template includes structural constraints, content constraints, and association constraints. The structural constraints require that the output JSON data must include four fields: a shot array, a role array, a scene array, and a props array. The content constraints require that the script field of each shot in the shot array includes shot type, camera movement, scene description, action, and dialogue, and that the character names use full names and prohibit the use of pronouns. The association constraints require that the role names referenced by the role field in the shot array must be names already defined in the role array.

[0023] During execution, the structured prompt template called by the business logic processing module first limits the core fields of the JSON data through structural constraints. It specifies that the four fields—shots (screenshot array), roles (roles array), scenes (scene array), and props (prop array)—are mandatory and cannot be empty. This ensures that the decomposed storyboard data contains the four core elements: storyboard, characters, scenes, and props, preventing data loss that could cause subsequent generation operations to fail. Then, it standardizes the specific content of the storyboard script field through content constraints, specifying the shot type (supporting four types: long shot, medium shot, close-up, and extreme close-up) and camera movement. (Supports five basic camera movement methods: push, pull, pan, tilt, and follow) and other key shooting parameters. At the same time, it requires the use of full names for characters, which can effectively avoid confusion in character identification caused by the use of pronouns. It ensures that the scene description, actions and dialogue of each shot are complete and accurate. The dialogue text must be consistent with the original script, with an error of no more than 1 character. Finally, it achieves precise binding between shots and characters through association constraints, ensuring that the character names referenced by the roles field in the shot array are all defined characters in the roles array, and that the character names are completely consistent in capitalization, avoiding generation chaos caused by undefined characters. Through the synergistic effect of the above three constraints, the structured JSON data generated by the large language model decomposition has a standardized format, complete content, and clear relationships. The JSON data uses UTF-8 encoding and is indented by 2 spaces, which facilitates the subsequent parsing and calling by the intelligent generation service module. This provides standardized and accurate data support for the subsequent intelligent generation service module to perform image and video generation operations, so that the generated storyboards and video content can strictly conform to the script logic. At the same time, it allows those skilled in the art to adjust the prompt word template according to the constraints to adapt to the structured decomposition needs of different types of scripts. This effectively solves the problem of the generated content being out of touch with the script due to non-standard script decomposition and messy data in the existing technology.

[0024] Furthermore, in step S2, after obtaining the JSON data returned by the large language model, the business logic processing module compares the newly generated role and scene names with the existing entities in the current project in the database through an entity matching algorithm. If the names match exactly or there is a variant match, the ID of the existing entity is reused to achieve consistent association of role images across episodes.

[0025] In specific execution, after receiving the structured JSON data returned by the large language model, the business logic processing module first extracts the newly generated role and scene name contained in the data, and then calls the entity matching algorithm (the matching threshold is set to 0.85, and cosine similarity is used for calculation) to retrieve the role and scene entity information stored in the current project from the database (MySQL 8.0 and above). The role entity table in the database contains fields such as id (primary key, varchar(32), generated using UUID), name (full role name, varchar(100)), age (age, int), identity (identity, varchar(50)), and createTime (creation time, datetime). The scene entity table contains fields such as id (primary key, varchar(32)), name (scene name, varchar(100)), and description (scene description, text). The newly generated name is compared one by one with existing entity names. During the comparison, both identical name matches and variant matches with attribute limitations (such as "Zhang San" and "Zhang San (teenager)") are identified. Once a match is detected (similarity ≥ 0.85), the ID of the existing entity is immediately reused to bind the newly generated character / scene to the existing entity. If no match is detected (similarity < 0.85), a unique ID (UUID format) is assigned to the new character / scene and stored in the database. This process, through an entity ID reuse mechanism, achieves unified association of characters and scene entities across episodes, avoiding the repeated creation of the same character as different entities in different episodes and scenes. This fundamentally solves the problem of inconsistent facial features and clothing details for the same character in different scenes in existing technologies, ensuring consistency in character image and scene style throughout the narrative. It also reduces database storage redundancy, lowers data storage costs, and facilitates later project maintenance and management. Those skilled in the art can adjust the comparison precision of the matching algorithm (adjustable from 0.7 to 0.95) to adapt to the entity matching needs of different scenarios, ensuring the accuracy and efficiency of the matching results.

[0026] Furthermore, the entity matching algorithm includes exact name matching and variant recognition. The variant matching is the matching of character names with attribute limitations, specifically including the matching forms of "full character name + age" and "full character name + identity".

[0027] During execution, the entity matching algorithm first performs exact name matching, comparing the newly generated role and scene names with existing entity names in the database to ensure complete consistency (case-sensitive and space-sensitive), quickly identifying identical entities that do not require attribute distinctions, ensuring efficient matching. For role names with attribute limitations, the variant recognition function is then used for comparison, focusing on identifying two common variant forms: "full role name + age" and "full role name + identity". Variant matching adopts a split comparison method of "core name + attribute", first extracting the core full name of the role, and then comparing the correlation of attribute fields. For example, "Zhang San" is identified as different variants of the same role as "Zhang San (teenager)" and "Zhang San (doctor)", achieving accurate matching between variants and the original entity. This algorithm combines exact matching with variant recognition, ensuring rapid matching of identical entities while resolving matching failures caused by attribute limitations in character names. It avoids misjudging characters based on changes in age or identity, further strengthening the consistency of character images across episodes. Simultaneously, the two specific variant matching formats provide clear reference standards for algorithm execution. Those skilled in the art can extend the variant matching formats (e.g., "full character name + profession") based on these standards and actual script requirements, improving the algorithm's adaptability and ensuring the comprehensiveness and accuracy of entity matching. This provides a reliable entity association foundation for subsequent generation of visually consistent characters, effectively reducing character image confusion caused by entity matching errors.

[0028] Furthermore, in step S3, the encapsulation and distribution process of the asynchronous task specifically includes: S31: The business logic processing module verifies the validity of the parameters of the AI-generated request. The parameters include the project ID and the target scene ID. After the verification is successful, an AiTask task entity is created in the database. The initial state of the AiTask task entity is set to pending. S32: The task producer of the task scheduling module pushes the AiTask task entity into the Redis message queue, the task consumer listens to the Redis message queue and obtains the task, and updates the status of the AiTask task entity to running. S33: The task consumer distributes the AiTask task entity to the processing unit corresponding to the intelligent generation service module according to the task type. The task type includes the storyboard image generation task shotImageGenerate and the video generation task videoGenerate.

[0029] In specific execution, the business logic processing module first verifies the validity of the core parameters of the AI-generated request (project ID, target storyboard ID). The project ID is a UUID in varchar(32) format, and the target storyboard ID is int format (≥1). After the verification passes (the parameter format is correct, and the project and storyboard exist), the AiTask task entity is created in the database. The AiTask task entity contains id (primary key, varchar(32), generated by UUID), projectId (project ID, varchar(32)), shotId (target storyboard ID, int), taskType (task type, varchar(50), with the value shotImageGenerate or videoGenerate), status (task status, varchar(20), initially pending), createTime (creation time, datetime), updateTime (update time, datetime). Fields such as `e` and `outputJson` (generated result URL, text) clearly define the project, storyboard, and generation requirements corresponding to the task. Subsequently, the task producer in the task scheduling module pushes the AiTask task entity into a Redis message queue. The queue is set to a maximum length of 10,000 messages to achieve temporary storage and queuing of tasks. Simultaneously, the task consumer (with 3-8 concurrent threads, adjustable according to hardware configuration) continuously listens to the Redis message queue. When a task awaiting execution is detected in the queue, it immediately retrieves the task and updates its status to "running," marking the task as being executed. Finally, based on the `taskType` field, the task consumer distributes storyboard image generation tasks (`shotImageGenerate`) to the image generation processing unit (deployed GPU, ≥16GB VRAM) of the intelligent generation service module, and distributes video generation tasks (`videoGenerate`) to the video generation processing unit (deployed GPU, ≥24GB VRAM), ensuring that different types of generation tasks are accurately assigned to the corresponding processing units. This process ensures the legitimacy of tasks through parameter verification, achieves orderly task scheduling through Redis message queues to avoid task congestion, and achieves precise allocation of generated resources through task type distribution, effectively avoiding inefficiency and resource waste caused by the mixed execution of different types of tasks. At the same time, real-time updates of task status facilitate users and the system to monitor task execution progress. This solves the problems of chaotic scheduling and opaque execution status of AI-generated tasks in existing technologies. Those skilled in the art can expand the task types and corresponding processing units according to actual generation needs to adapt to the scheduling needs of AI-generated tasks in more scenarios.

[0030] Furthermore, in step S3, the task scheduling module sets differentiated timeout thresholds for different types of asynchronous tasks, with the timeout threshold for video generation tasks set to 15 minutes and the timeout threshold for ordinary tasks set to 5 minutes; if a task is not completed within the specified timeout threshold, the monitoring service forcibly sets the status of the task to failure and releases system resources.

[0031] During execution, the task scheduling module first presets differentiated timeout thresholds for different types of asynchronous tasks based on their time consumption characteristics. For example, video generation tasks, which involve multi-frame image stitching and dynamic effect rendering, take a relatively long time, so the timeout threshold is set to 15 minutes (±1 minute error). For ordinary tasks such as storyboard image generation, which take a shorter time, the timeout threshold is set to 5 minutes (±1 minute error), ensuring that the timeout threshold matches the actual task consumption. At the same time, the system's monitoring service (using scheduled tasks with a detection frequency of 10 seconds / time) continuously monitors all AiTask tasks in the running state and calculates the execution time of each task in real time. When the execution time of a task exceeds its corresponding timeout threshold and is still not completed, the monitoring service immediately triggers the exception handling mechanism, forcibly setting the status of the task to failure (status set to failed), and releasing the model computing resources and memory resources occupied by the task. The release delay does not exceed 30 seconds to avoid resources being occupied by unfinished tasks for a long time. This process avoids the problems of video generation tasks being misjudged as failed and ordinary tasks occupying resources for too long due to uniform timeout thresholds. It also achieves efficient utilization of system resources through anomaly handling of monitoring services, prevents system lag caused by resource blockage, and ensures the stability and smoothness of the entire AI generation process. Those skilled in the art can adjust the timeout thresholds of different tasks (adjustable from 5 to 30 minutes for video generation tasks and 1 to 10 minutes for ordinary tasks) according to actual conditions such as hardware configuration (CPU ≥ 8 cores, memory ≥ 16GB) and model performance to further optimize the efficiency of system resource allocation.

[0032] Furthermore, in step S4, when the intelligent generation service module performs the image generation operation, it receives the character reference image or character feature LoRA model passed in by the business logic processing module, and combines it with the prompt words of the current storyboard to generate a fusion image, so as to ensure the visual consistency of the same character in different storyboard scenes.

[0033] In practice, after receiving the image generation task distributed by the task scheduling module, the intelligent generation service module first obtains the corresponding structured JSON data from the business logic processing module, extracts the prompts for the current storyboard (including scene descriptions, character actions, etc.), and controls the prompt length to 50-500 characters. Simultaneously, it receives a reference image (resolution 1024×1024-4096×4096 pixels, PNG / JPG format) or a character feature LoRA model (weights 0.6-0.8, training steps ≥1000 steps) from the business logic processing module, and then... Visual features (extracted via CNN, 512-dimensional), character feature parameters from the LoRA model, and storyboard prompts are fused together, with a fusion weight of 0.5 for prompts and 0.5 for character features. This generates a complete generation instruction adapted to the current storyboard scene. This instruction is then passed to an image generation model (e.g., StableDiffusion 1.5 / 2.1, 20-50 inference steps, CFGScale set to 7-10) to perform the storyboard image generation operation. The generated image resolution supports 1080P / 4K, and the generation time per frame does not exceed 10 seconds (GPU memory ≥16GB). By fusing character reference images or the LoRA model with storyboard prompts, the image generation model can accurately capture the character's facial features, clothing details, expressions, and other core visual information, ensuring that the generated character image is consistent with the preset character visual settings. Even in different storyboards and scenes, the core visual features of the same character will not change. This process effectively solves the problem in existing technologies where image generation tools cannot perceive the context of the script, resulting in inconsistent visual appearances of the same character. It ensures the visual coherence of continuous narrative. At the same time, the dual selection of character reference images and LoRA models caters to the needs of different users. Those skilled in the art can choose the appropriate character visual setting method according to the complexity of the character to ensure the accuracy of the generated effect.

[0034] In this embodiment, in step S2, when generating structured JSON data, the business logic processing module simultaneously parses the script dialogue and action descriptions to obtain the duration of the character's action corresponding to each scene, and embeds the duration of the character's action as a hidden field into the character list field of the structured JSON data; in step S4, when the intelligent generation service module calls the AI ​​generation model to generate character action scenes, it extracts the hidden character action duration field and reuses it as a frame interval control parameter of the AI ​​generation model to regulate the frame interval of the output scene of the AI ​​generation model.

[0035] During execution, the business logic processing module, while driving the large language model to generate structured JSON data, simultaneously performs semantic parsing on the dialogue content and character action descriptions in the script. It employs a BERT-based semantic parsing model (training steps ≥ 50,000), combining factors such as action complexity (divided into simple, medium, and complex levels with corresponding weights of 0.3, 0.5, and 0.7), and dialogue length (0.5 seconds of base duration for every 10 characters) to calculate the duration of each character's action in each scene. For example, "slowly raising a hand" (complex action, no dialogue) is parsed as a 3-second duration, "quickly turning around" (simple action, no dialogue) as a 1-second duration, and "speaking softly and nodding slowly" (medium action, 20 characters of dialogue) as a 2.5-second duration. This duration is then used as a hidden field (field name: actionDur). The hidden action duration (data type: float, unit: seconds, rounded to one decimal place) is embedded in the corresponding role entry of the role list in the structured JSON data. It does not affect the normal parsing and use of the JSON data. When the intelligent generation service module performs the action screen generation operation, it extracts the hidden action duration from the role list field of the structured JSON data and reuses it directly as the frame interval control parameter of the AI ​​generation model. The frame interval calculation logic is "frame interval = action duration / target number of frames". The target number of frames is set to 10 frames / second by default (configurable to 5-30 frames / second). For example, a 3-second duration corresponds to a frame interval setting of 30 frames (10 frames / second) (frame interval 0.1 seconds), a 1-second duration corresponds to a frame interval setting of 10 frames (frame interval 0.1 seconds), and a 2.5-second duration corresponds to a frame interval setting of 25 frames (frame interval 0.1 seconds). This process obtains the duration of actions through script parsing and reuses it as a frame interval parameter. Without the need for additional frame interval adjustment modules or model training, it solves the problem of mismatch between the duration of character actions and script descriptions caused by fixed frame intervals in existing AI generation models. This ensures that the speed and duration of generated character actions are completely consistent with the script settings, improving the accuracy of the generated content. Those skilled in the art can adapt different types of action descriptions by adjusting the parsing logic of action duration (such as adjusting the action complexity weight and dialogue duration conversion ratio) to ensure the accuracy of the parsing results.

[0036] In step S2, when the business logic processing module generates structured JSON data, it simultaneously converts the scene lighting and shadow description of the storyboard into a grayscale threshold parameter, which is then embedded as an additional field into the scene list field of the structured JSON data. In step S4, when the intelligent generation service module calls the AI ​​generation model to generate scene images, it reuses the grayscale threshold parameter as the lighting and shadow rendering threshold of the AI ​​generation model to control the brightness and darkness of the scene images.

[0037] In practice, during the generation of structured JSON data, the business logic processing module synchronously extracts the scene lighting descriptions (such as "dim study" and "bright courtyard") for each scene. Using a semantic mapping algorithm (trained based on Word2Vec, with a vector dimension of 256), the lighting descriptions are converted into quantifiable grayscale threshold parameters (grayscale values ​​ranging from 0 to 255, integers). Specifically, "dim" scenes correspond to grayscale thresholds of 80-120, "bright" scenes to 180-220, and "medium brightness" scenes to 120-180. For example, "dim study" corresponds to a grayscale threshold of 80-120, "bright courtyard" to 180-220, and "afternoon living room" to 140-160. This grayscale threshold parameter is then... The number is used as an additional field (field name: grayThreshold, data type: int[], storage threshold range upper and lower limits), which is embedded in the scene list of the structured JSON data under the corresponding scene entry to realize the association storage of light and shadow information and scene data. After receiving the scene image generation task, the intelligent generation service module extracts the grayscale threshold parameter from the scene list field of the structured JSON data and reuses it as the light and shadow rendering threshold of the AI ​​generation model. When the model (such as StableDiffusion) renders the scene image, it controls the grayscale value of the image within the threshold range, where the grayscale value of the dark part is not lower than the lower limit of the threshold and the grayscale value of the bright part is not higher than the upper limit of the threshold, thereby realizing the precise control of the brightness of light and shadow, so that the generated scene light and shadow are highly consistent with the script description. This process converts scene lighting descriptions into grayscale thresholds and reuses them as rendering thresholds, eliminating the need for additional lighting rendering control modules. This solves the problem of scene lighting and shadows not matching the script description caused by random lighting and shadow rendering in existing technologies, thus improving the accuracy and fit of scene generation. Those skilled in the art can optimize the matching degree between grayscale thresholds and lighting and shadow descriptions by adjusting the parameters of the semantic mapping algorithm (such as vector dimension and mapping threshold), adapting to the lighting and shadow generation needs of different scenes.

[0038] Example 2 An AI video content integrated generation system based on script structure is used in the AI ​​video content integrated generation method based on script structure described in Embodiment 1. The system includes a user interaction module, a business logic processing module, a task scheduling module, and an intelligent generation service module. Each module establishes a connection through a preset protocol to form a data closed loop. The user interaction module runs on the browser side, providing a script editing interface, a storyboard list view, and a multimedia editor. It integrates a status management unit to maintain the context data of the current project, episode, and storyboard. The business logic processing module runs in a Node.js environment, integrates the PrismaORM database adapter, and includes a storyboard service unit. It is used to receive story text input by the user, call the large language model through structured prompt word templates to generate structured JSON data, and perform entity matching operations. The task scheduling module is built on Redis message queue BullMQ and includes task producers and task consumers, which are used to encapsulate asynchronous AI task generation, task distribution and task status updates. The intelligent generation service module runs in the Python FastAPI environment. As a model gateway, it encapsulates the calling interfaces of different AI models to perform text, image and video generation operations and achieve visual consistency generation of characters.

[0039] When the system is executed, the user interaction module runs on the browser (compatible with Chrome 100.0 and above, Edge 100.0 and above), providing users with a visual script editing interface, storyboard list view, and multimedia editor. Its internal state management unit (using the Vuex / Pinia framework) maintains the context data of the current project, episode, and storyboard in real time, with a data update latency of no more than 500ms, ensuring the continuity of user operations and data consistency. The business logic processing module runs on Node.js 16.x and above, achieving efficient interaction with the database (MySQL 8.0 and above) through the integrated PrismaORM 4.x and above database adapter, with an interaction response time of no more than 300ms. Its included storyboard service unit receives the story text input by the user, calls the structured prompt word template to drive the large language model to generate structured JSON data, and performs entity matching operations. The system parses and associates data with a parsing efficiency of at least 1000 characters per second. The task scheduling module, built on Redis 6.2.x and above, and BullMQ 4.x and above, works collaboratively with task producers and consumers (3-8 concurrent threads) to encapsulate, queue, distribute, and update the status of AI-generated tasks. Task distribution latency is no more than 1 second, ensuring orderly task execution. The intelligent generation service module runs on Python 3.9.x and above, and FastAPI 0.100.0 and above. As a model gateway, it encapsulates the calling interfaces for various AI models (GeminiPro, StableDiffusion, Runway), with an interface response time of no more than 500ms. It receives tasks distributed by the task scheduling module, combines structured JSON data and character visual settings, and performs text, image, and video generation operations. A fusion generation mechanism ensures visual consistency for the characters. Each module establishes a connection through a preset protocol (HTTP 1.1 / WebSocket), forming a complete data loop. Data flows automatically between modules without manual user intervention. The minimum hardware configuration requirements are CPU ≥ 8 cores, memory ≥ 16GB, and GPU memory ≥ 16GB. It can support 10-50 concurrent tasks, which not only solves the problems of module fragmentation and poor data link in existing technologies, but also realizes integrated generation from script to video through the functional collaboration of each module. Those skilled in the art can adjust the operating environment and configuration parameters of each module according to actual deployment needs to ensure stable system operation and adaptability to different generation requirements.

[0040] Furthermore, the user interaction module and the business logic processing module establish a connection via the HTTP / WebSocket protocol for sending creation instructions and receiving generation results in real time; the business logic processing module is connected to the task scheduling module for transmitting AI generation requests; the task scheduling module is connected to the intelligent generation service module for driving the underlying AI model to perform specific generation operations; the intelligent generation service module uploads the generation results to the object storage OSS, and then sends them back to the user interaction module sequentially through the task scheduling module and the business logic processing module.

[0041] During execution, the user interaction module and the business logic processing module establish bidirectional communication via the HTTP 1.1 / WebSocket protocol. HTTP is used to transmit creation instructions (request timeout set to 30 seconds), while WebSocket is used to transmit generation results and task status in real time (connection timeout set to 60 seconds). Creation instructions such as script editing and generation requests issued by the user on the front-end interface are quickly transmitted to the business logic processing module via HTTP, with a transmission latency of no more than 300ms. Meanwhile, information such as intelligent generation results and task execution status are transmitted back to the front-end interface in real time via WebSocket, with a transmission frequency of once per second, ensuring that users can view the generation progress and results in real time and improving the interactive experience. The business logic processing module transmits the parsed AI generation requests to the task scheduling module. The requests are encapsulated in JSON format, with a data size not exceeding 10MB, ensuring that the task... The system accurately transmits task requirements, providing clear instruction support for task scheduling. After receiving a request, the task scheduling module encapsulates it into an asynchronous task and distributes it to the intelligent generation service module, driving the underlying AI model to execute specific generation operations. The task distribution delay is no more than 1 second, achieving efficient linkage between task scheduling and generation execution. After completing the image and video generation, the intelligent generation service module uploads the generated results to Object Storage Service (OSS) (compatible with Alibaba Cloud OSS and Tencent Cloud COS), with the storage path "projectId / shotId / generation type / timestamp.format", avoiding the generation results from occupying local system resources. At the same time, it obtains the URL of the generated results (valid for 7 days, configurable) and sends it back to the task scheduling module. The task scheduling module then transmits the URL and task status to the business logic processing module, which finally synchronizes it to the user interaction module for users to view, download, or further edit. This connection method and data transmission path ensure the stability, real-time performance, and efficiency of communication between modules, achieving end-to-end connectivity for instruction sending, request transmission, result generation, and feedback. It solves the problems of poor inter-module communication and data transmission delays in existing technologies. Those skilled in the art can adjust the connection protocol between modules (such as upgrading to HTTP / 2) according to actual communication needs to ensure the security and stability of data transmission.

[0042] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Unless otherwise specified, an element defined by the phrase "comprising..." or "including..." does not exclude the presence of additional elements in the process, method, article, or terminal device that includes said element. Additionally, in this document, "greater than," "less than," "exceeding," etc., are understood to exclude the stated number; "above," "below," "within," etc., are understood to include the stated number.

[0043] The above description of the embodiments is provided to facilitate understanding and use of the present invention by those skilled in the art. It is obvious to those skilled in the art that various modifications can be easily made to the embodiments, and the general principles described herein can be applied to other embodiments without creative effort. Therefore, the present invention is not limited to the above embodiments. Improvements and modifications made by those skilled in the art based on the disclosure of the present invention without departing from the scope of the present invention should be within the protection scope of the present invention.

Claims

1. A script-structured AI video content integrated generation method, characterized in that, Includes the following steps: S1: Receives story text in unstructured natural language text format from the user through the user interaction module; S2: The business logic processing module receives the story text, calls the preset structured prompt word template, and drives the large language model to automatically decompose the story text into structured JSON data containing shot numbers, scene descriptions, character lists, and scene lists; S3: The task scheduling module encapsulates the AI ​​generation request corresponding to the structured JSON data into an asynchronous task and distributes it to the intelligent generation service module according to the task type; S4: The intelligent generation service module performs text, image, or video generation operations based on the structured JSON data and the associated character visual settings to obtain the generation results; S5: The task scheduling module receives the generated result, updates the task status, and sends it back to the business logic processing module, which then synchronizes it to the front-end interactive interface.

2. The AI ​​video content integrated generation method based on script structuring as described in claim 1, characterized in that, In step S2, the structured prompt template includes structural constraints, content constraints, and association constraints. The structural constraints require that the output JSON data must contain four fields: a shot array, a role array, a scene array, and a props array. The content constraints require that the script field of each shot in the shot array includes shot type, camera movement, scene description, action, and dialogue, and that the role names use full names and prohibit the use of pronouns. The association constraints require that the role names referenced by the role field in the shot array must be names already defined in the role array.

3. The AI ​​video content integrated generation method based on script structuring as described in claim 1, characterized in that, In step S2, after obtaining the JSON data returned by the large language model, the business logic processing module compares the newly generated role and scene names with the existing entities in the current project in the database through the entity matching algorithm. If the names match exactly or there is a variant match, the ID of the existing entity is reused to achieve consistent association of role images across episodes.

4. The AI ​​video content integrated generation method based on script structuring according to claim 3, characterized in that, The entity matching algorithm includes exact name matching and variant recognition. The variant matching is the matching of character names with attribute limitations, specifically including the matching forms of "full character name + age" and "full character name + identity".

5. The AI ​​video content integrated generation method based on script structuring according to claim 1, characterized in that, In step S3, the encapsulation and distribution process of the asynchronous task specifically includes: S31: The business logic processing module verifies the validity of the parameters of the AI-generated request. The parameters include the project ID and the target scene ID. After the verification is successful, an AiTask task entity is created in the database. The initial state of the AiTask task entity is set to pending. S32: The task producer of the task scheduling module pushes the AiTask task entity into the Redis message queue, the task consumer listens to the Redis message queue and obtains the task, and updates the status of the AiTask task entity to running. S33: The task consumer distributes the AiTask task entity to the processing unit corresponding to the intelligent generation service module according to the task type. The task type includes the storyboard image generation task shotImageGenerate and the video generation task videoGenerate.

6. The AI ​​video content integrated generation method based on script structuring according to claim 5, characterized in that, In step S3, the task scheduling module sets differentiated timeout thresholds for different types of asynchronous tasks. The timeout threshold for video generation tasks is set to 15 minutes, and the timeout threshold for ordinary tasks is set to 5 minutes. If a task is not completed within the specified timeout threshold, the monitoring service forces the task to be set to failure and releases system resources.

7. The AI ​​video content integrated generation method based on script structuring according to claim 1, characterized in that, In step S4, when the intelligent generation service module performs the image generation operation, it receives the character reference image or character feature LoRA model from the business logic processing module, and combines it with the prompt words of the current storyboard to generate a fusion image, so as to ensure the visual consistency of the same character in different storyboard scenes.

8. The AI ​​video content integrated generation method based on script structuring according to claim 1, characterized in that, In step S2, when generating structured JSON data, the business logic processing module synchronously parses the script dialogue and action descriptions to obtain the duration of the character's action corresponding to each scene, and embeds the duration of the character's action as a hidden field into the character list field of the structured JSON data. In step S4, when the intelligent generation service module calls the AI ​​generation model to generate character action scenes, it extracts the hidden character action duration field and reuses it as the frame interval control parameter of the AI ​​generation model to adjust the frame interval of the output screen of the AI ​​generation model.

9. An AI video content integrated generation system based on script structuring, characterized in that, To implement the AI ​​video content integrated generation method based on script structure as described in any one of claims 1-8, the system includes a user interaction module, a business logic processing module, a task scheduling module, and an intelligent generation service module. Each module establishes a connection through a preset protocol to form a data closed loop. The user interaction module runs on the browser side, providing a script editing interface, a storyboard list view, and a multimedia editor. It integrates a status management unit to maintain the context data of the current project, episode, and storyboard. The business logic processing module runs in a Node.js environment, integrates the PrismaORM database adapter, and includes a storyboard service unit. It is used to receive story text input by the user, call the large language model through structured prompt word templates to generate structured JSON data, and perform entity matching operations. The task scheduling module is built on Redis message queue BullMQ and includes task producers and task consumers, which are used to encapsulate asynchronous AI task generation, task distribution and task status updates. The intelligent generation service module runs in the Python FastAPI environment. As a model gateway, it encapsulates the calling interfaces of different AI models to perform text, image and video generation operations and achieve visual consistency generation of characters.

10. The AI ​​video content integrated generation system based on script structuring according to claim 9, characterized in that, The user interaction module and the business logic processing module establish a connection via the HTTP / WebSocket protocol for sending creation instructions and receiving generation results in real time. The business logic processing module is connected to the task scheduling module for transmitting AI generation requests. The task scheduling module is connected to the intelligent generation service module for driving the underlying AI model to perform specific generation operations. The intelligent generation service module uploads the generation results to the object storage OSS, and then sends them back to the user interaction module sequentially through the task scheduling module and the business logic processing module.

Images