Short play work batch generation method and device, and electronic equipment

Abstract

This invention provides a method, apparatus, and electronic device for batch generation of short drama works, belonging to the field of film and television work generation technology. The method includes: a master node acquiring the script of the short drama to be generated; decomposing the script into multiple atomic tasks and determining the dependencies between these tasks; based on the dependencies, distributing atomic tasks without dependencies to multiple slave nodes in a distributed generation system in parallel via a message queue; each slave node executing the received atomic tasks in parallel, writing the intermediate data generated as execution results into a distributed storage system; and the master node reading the execution results of each atomic task from the distributed storage system, merging them according to the dependencies and the time sequence of the script to generate the short drama video. This invention enables high-concurrency batch production of short dramas, significantly improving the efficiency and stability of short drama generation.

Description

Technical Field

[0001] This invention relates to the field of film and television production technology, specifically to a method, apparatus, and electronic device for batch production of short dramas. Background Technology

[0002] Short dramas (also known as micro-dramas or vertical-screen dramas) refer to online film and television works with episode lengths typically ranging from a few seconds to fifteen minutes, featuring a continuous storyline and a clear theme. Current short drama production methods primarily rely on traditional film and television production processes, involving manual completion of scriptwriting, storyboard design, on-set filming, post-production editing, and dubbing. Each step is executed sequentially, and the production cycle for a single short drama often lasts for weeks or even months. To improve efficiency, some solutions have introduced Artificial Intelligence Generated Content (AIGC) technology, which automates the generation of short dramas through multiple intelligent agents (such as story-generating agents, storyboard-generating agents, and dubbing-generating agents) in a streamlined manner.

[0003] However, in the process of realizing this invention, the inventors discovered that the existing methods have at least the following drawbacks: First, existing AIGC short drama generation solutions typically adopt a single-process serial or simple parallel pipeline architecture, with each agent executing sequentially. This results in long generation times for a single episode, making it impossible to support the mass production needs of thousands or even tens of thousands of episodes.

[0004] Second, existing solutions lack a fine-grained scheduling mechanism for heterogeneous computing resources (such as graphics processing units (GPUs) and central processing units (CPUs), resulting in extensive resource allocation and high production costs.

[0005] Third, in long-cycle, high-volume production processes, existing solutions require re-execution from the beginning if any link fails (such as node downtime or task timeout), resulting in poor system stability.

[0006] Fourth, in existing solutions, input / output (I / O) can easily become a bottleneck during high-concurrency read and write operations, and task status is difficult to synchronize across nodes.

[0007] Fifth, during the batch generation process, it is difficult to ensure the consistency of character images and scene styles between different storyboards and different episodes, which can easily lead to continuity errors such as sudden changes in character appearance and inconsistent background styles, seriously affecting the quality of the final product.

[0008] Therefore, how to build an automated production method that can support the mass parallel production of short dramas, achieve efficient scheduling of heterogeneous computing power, have complete fault tolerance and recovery capabilities, and ensure the consistency of generated content across storyboards is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0009] In view of this, it is necessary to provide a method, device and electronic device for batch generation of short drama works to solve the technical problems of low generation efficiency and poor consistency of cross-scene content caused by the serial or simple pipeline architecture of existing short drama production methods.

[0010] To address the aforementioned technical problems, in a first aspect, the present invention provides a method for batch generation of short drama works, applied to a distributed generation system, wherein the distributed generation system includes a master node and at least two slave nodes, and the method for batch generation of short drama works includes: The master node obtains the script for the short drama to be generated; The master node breaks down the short drama script into multiple atomic tasks and determines the dependencies between each atomic task. The atomic tasks include image generation, video generation, audio generation, and synthesis tasks. The master node distributes atomic tasks that do not have dependencies to multiple slave nodes in the distributed generation system in parallel through a message queue, based on the dependency relationship. Each slave node executes the received atomic tasks in parallel and writes the intermediate data generated during execution into the distributed storage system as the execution result; The master node reads the execution results of each atomic task from the distributed storage system, merges them according to the dependencies and the time sequence of the short drama script, and generates a short drama video.

[0011] In one possible implementation, the master node decomposes the short drama script into multiple atomic tasks, including: The large language model is used to extract character information, scene information, shot information, dialogue text, and shot duration information from the short drama script; The tasks are broken down according to the hierarchy of episodes and storyboards, and each storyboard corresponds to four atomic tasks, including an image generation task, a video generation task, an audio generation task, and a compositing task. Metadata is generated for each storyboard, including character information, scene information, shot information, dialogue text, and storyboard duration information. The metadata is then written into the distributed storage system, wherein each atomic task corresponding to the same storyboard is associated with the metadata corresponding to that storyboard.

[0012] In one possible implementation, the master node determines the dependencies between atomic tasks, including: Construct a directed acyclic graph, and label the plot continuity dependencies between scenes based on the directed acyclic graph; Specifically, when the last frame of the previous shot in a continuous storyboard is used as the first frame of the next shot, a dependency relationship is marked between the video generation tasks corresponding to the two shots; no dependency relationship exists between the other atomic tasks.

[0013] In one possible implementation, the step of distributing atomic tasks with no dependencies to multiple slave nodes in a distributed cluster in parallel via message queues includes: Obtain the computing power utilization and computing power level of each slave node; Based on the task type weight and business priority of the atomic task, as well as the computing power utilization and computing power level of each slave node, calculate the scheduling cost of each atomic task relative to each slave node. For each atomic task, the slave node with the lowest scheduling cost is selected as the target execution node for that atomic task, and the atomic task is distributed to the target execution node.

[0014] In one possible implementation, calculating the scheduling cost of each atomic task relative to each slave node includes: The scheduling cost is calculated using the following formula. Cost ( i , j ):

[0015] in, T i The task type weight for atomic tasks. P i The business priority of atomic tasks, U j For the node j computing power utilization rate C j For the node j The computing power level, α and β For preset weighting coefficients, ε It is a very small positive number.

[0016] In one possible implementation, the master node reads the execution results of each atomic task from the distributed storage system, merges them according to the dependencies and the chronological order of the short drama script, and generates a short drama video, including: Read the character feature vectors of the same character from the metadata corresponding to different scenes in the distributed storage system, and calculate the cosine similarity between the feature vectors of each character; When the cosine similarity is less than or equal to a preset threshold, it is determined that there is an inconsistency in the character image of the corresponding storyboard, and the regeneration of the atomic task corresponding to the storyboard is automatically triggered. When the cosine similarity of all scenes is greater than the preset threshold, the execution results of each atomic task are read from the distributed storage system according to the scene sequence. The execution results include the synthesized segments corresponding to each scene. The pre-composite segments corresponding to each scene are spliced ​​together to generate a short drama video file and output it.

[0017] In one possible implementation, the method for batch generation of short drama works includes: The master node monitors the heartbeat signals of each slave node and the execution time of atomic tasks; When the master node fails to receive a heartbeat signal from the slave node for a preset number of consecutive times, it determines that the slave node is faulty, reallocates the unfinished atomic tasks on the slave node to other slave nodes, and instructs the newly allocated slave nodes to read the checkpoint data of the atomic tasks from the distributed storage system to continue execution. When the execution time of an atomic task exceeds a preset timeout threshold, the master node determines that the atomic task has failed and triggers automatic retry. If the number of retries exceeds a preset threshold, it will perform a downgrade or issue an alarm.

[0018] In one possible implementation, the consistency of character images in the generated content is constrained by character feature vectors in the image generation task and / or video generation task. The character feature vector is generated in the following way: the character image features are extracted using the CLIP or EVA-CLIP model, mapped to a preset dimension through a fully connected network, and ReLU is used as the activation function. When performing the image generation task, the character feature vector is concatenated to the text embedding layer of the text-generated image model; When performing the video generation task, the keyframes output by the image generation task are read from the distributed storage system as input frames.

[0019] On the other hand, the present invention also provides a device for mass-producing short drama works, comprising: The data acquisition module is used to acquire the script of the short drama to be generated; The task decomposition module is used to decompose the short drama script into multiple atomic tasks and determine the dependencies between each atomic task. The atomic tasks include image generation tasks, video generation tasks, audio generation tasks, and synthesis tasks. The parallel distribution module is used to distribute atomic tasks that do not have dependencies to multiple slave nodes in the distributed generation system in parallel through a message queue, based on the dependencies. The parallel generation module is used to execute the received atomic tasks in parallel and write the intermediate data generated during execution into the distributed storage system as the execution result. The short drama synthesis module is used to read the execution results of each atomic task from the distributed storage system, merge them according to the dependencies and the time sequence of the short drama script, and generate a short drama video.

[0020] Thirdly, the present invention also provides an electronic device, including a memory and a processor, wherein, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the short drama production batch generation method described in any of the above implementations.

[0021] The beneficial effects of this invention are as follows: The method for batch generation of short drama works provided by this invention, after obtaining the short drama script through the master node, first breaks down the script into several atomic tasks, including image generation, video generation, audio generation, and compositing, while clarifying the dependencies between tasks. For atomic tasks without dependencies, they are directly distributed to multiple slave nodes for execution simultaneously through a message queue. Each slave node processes the received task in parallel, and the generated intermediate results are uniformly stored in distributed storage, realizing parallel processing by multiple slave nodes and improving the generation efficiency of short drama works. After all tasks are completed, the master node retrieves the execution results from the storage and assembles them into a complete short drama video according to the original dependencies and script order. This transforms serial processing into parallel processing, and the collaboration between multiple nodes based on distributed storage and message queues significantly improves the overall efficiency of short drama batch production. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 A schematic flowchart of an embodiment of the method for batch generation of short drama works provided by the present invention; Figure 2 For the present invention Figure 1 A schematic diagram of an embodiment of S102; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 4 For the present invention Figure 1 A schematic diagram of an embodiment of S105; Figure 5 A schematic flowchart of another embodiment of the method for batch generation of short drama works provided by the present invention; Figure 6 A schematic flowchart of another embodiment of the method for batch generation of short drama works provided by the present invention; Figure 7 A schematic diagram of an embodiment of the short drama production batch generation device provided by the present invention; Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0024] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0025] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0026] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] This invention provides a method, apparatus, and electronic device for batch generation of short drama works. The technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0029] Figure 1This is a schematic flowchart of an embodiment of the short drama batch generation method provided by the present invention. The short drama batch generation method is applied to a distributed generation system, which includes a master node and at least two slave nodes. The short drama batch generation method includes: S101, The main node obtains the short drama script to be generated; S102. The master node decomposes the short drama script into multiple atomic tasks and determines the dependencies between each atomic task. The atomic tasks include image generation task, video generation task, audio generation task, and synthesis task. S103. Based on the dependency relationship, the master node distributes atomic tasks that do not have a dependency relationship to multiple slave nodes in the distributed generation system in parallel through the message queue. S104. Each slave node executes the received atomic tasks in parallel and writes the intermediate data generated during execution into the distributed storage system as the execution result. S105. The master node reads the execution results of each atomic task from the distributed storage system, merges them according to the dependency relationship and the time sequence of the short drama script, and generates a short drama video.

[0030] The short drama production batch generation method provided in this embodiment is applied to a distributed generation system, which includes one master node and at least two slave nodes. The master node is internally divided according to functional logic and may include a task splitting module, a computing power scheduling module, a fault tolerance and checkpointing module, and a merging and verification module. The distributed storage system in this embodiment is implemented using the Ceph distributed file system, and the message queue uses the Kafka message queue. Slave nodes report task progress to the master node every 10 seconds and save checkpoint data to Ceph storage every 30 seconds.

[0031] The specific data flow is as follows: the task splitting module breaks down the script into atomic tasks and then passes the task information to the computing power scheduling module; the computing power scheduling module determines the task allocation scheme based on the real-time status of each slave node and distributes the atomic tasks to each slave node through the Kafka message queue; each slave node executes the tasks in parallel, and the intermediate data and execution results generated during the execution process are written to the Ceph distributed storage system in real time; the fault tolerance and checkpoint module of the master node continuously monitors the task execution status, and reads checkpoint data from Ceph for recovery in case of anomalies; after all tasks are completed, the merging and verification module reads the composite results of each scene from Ceph and completes the final merging and output. Throughout the entire process, data interaction between the master node and slave nodes, and between each module and Ceph, is conducted through standardized interfaces.

[0032] Atomic tasks include at least four types: image generation tasks, video generation tasks, audio generation tasks, and synthesis tasks.

[0033] In this context, dependency refers to the sequential constraint that one atomic task can only begin execution after another atomic task has been completed. This dependency is determined by the narrative continuity between storyboards. When the last frame of one storyboard in a continuous narrative needs to be used as the first frame of the next storyboard, a dependency exists between the video generation tasks corresponding to these two storyboards. That is, the video generation task for the later storyboard must wait for the video generation task for the earlier storyboard to complete before it can begin execution. Apart from this, there are no dependencies between image generation tasks, audio generation tasks, compositing tasks, or tasks of different types.

[0034] Intermediate data refers to the stage output generated by each slave node during the execution of atomic tasks, before final merging. The types of intermediate data generated by different atomic tasks vary: image generation tasks output storyboard sequence frames (i.e., keyframe images); video generation tasks output storyboard video clips without audio; audio generation tasks output MP3 audio files; and compositing tasks output complete storyboard clips containing video footage, audio tracks, and subtitles. This intermediate data, upon completion of the atomic task, is written uniformly into the Ceph distributed storage system as the execution result.

[0035] It's important to note that intermediate data refers to data relative to the final short drama; these are independent products that can be used in subsequent stages. For example, a composite clip of a storyboard is the final result for that storyboard, but it's still intermediate data for the entire short drama. It needs to be spliced ​​together with composite clips from other storyboards to form a complete short drama video. All intermediate data is organized in Ceph storage according to episode ID, storyboard number, and task type, facilitating subsequent task retrieval and master node merging.

[0036] In this way, the originally serially executed production process is broken down into independent task units that can be processed in parallel, with multiple slave nodes working simultaneously, thereby greatly improving the overall efficiency of short drama mass production.

[0037] In some embodiments of the present invention, such as Figure 2 As shown, in step S102, the master node breaks down the short drama script into multiple atomic tasks, including: S201. Use a large language model to extract character information, scene information, shot information, dialogue text and shot duration information from short drama scripts; S202. According to the hierarchy of episodes and storyboards, each storyboard is divided into four atomic tasks, including an image generation task, a video generation task, an audio generation task, and a compositing task. S203. Generate corresponding metadata for each storyboard, including character information, scene information, shot information, dialogue text and storyboard duration information, and write the metadata into a distributed storage system. Each atomic task corresponding to the same storyboard is associated with the metadata corresponding to that storyboard.

[0038] This embodiment uses a large language model as the core tool for script understanding. The following shows the specific implementation of how the main node decomposes the short drama script into multiple atomic tasks.

[0039] Specifically, the main node uses a large language model to extract information such as character style, scene information, shot information, dialogue text, and shot duration from the short drama script, and at the same time generates configuration parameters, such as 9:16 vertical screen ratio, 30fps frame rate, and TTS timbre.

[0040] For example, a user-submitted short drama script might be a story about "a boy's adventure in another world." The large language model will automatically extract the protagonist's physical characteristics, personality traits, different scenes where the story takes place (such as forests, castles, and dark caves), the composition and movement of each shot, the content of the characters' dialogue, and how many seconds each shot should last.

[0041] Furthermore, the main node is broken down level by level according to the hierarchy of "episode → storyboard". Each episode is divided into N independent storyboards, and the duration of each storyboard is usually controlled between 1 and 5 seconds. Each storyboard corresponds to the generation of four atomic tasks, namely an image generation task (IMAGE), a video generation task (VIDEO), an audio generation task (AUDIO), and a compositing task (CLIP).

[0042] Preferably, the image generation task is performed using a text-to-image model, the video generation task is performed using a text-to-image-to-video model (e.g., the Wan model), and the audio generation task is performed using the Qwen3-TTS speech synthesis model.

[0043] Furthermore, the master node generates a unique identifier for each atomic task. Preferably, the unique identifier can be in the format of "episode ID_storyboard number_task type_timestamp", such as "EP101_SC05_IMAGE_202405201430".

[0044] Meanwhile, the master node generates corresponding metadata for each scene, which includes information such as character ID, 1024-dimensional character feature vector, scene parameters, camera parameters, dialogue text, scene duration, and dependent task ID, and writes this metadata into the Ceph distributed storage system.

[0045] It should be understood that the four atomic tasks of image generation, video generation, audio generation, and synthesis under the same storyboard are all associated with the metadata corresponding to that storyboard. In this way, when a slave node executes an atomic task, it can quickly find the required parameters such as characters and scenes through the association.

[0046] Regarding the generation method of the character feature vector, specifically, this embodiment uses a CLIP or EVA-CLIP pre-trained visual model to extract the image features of the character, then feeds these features into a fully connected layer, maps them to 1024 dimensions, and uses the ReLU function as the activation function to finally obtain a feature vector that can characterize the visual identity of the character. This vector will subsequently be used to constrain the consistency of the character's image during the image and video generation process.

[0047] This embodiment ensures that different types of tasks within the same storyboard use consistent creative constraints through standardized metadata management and task identification mechanisms, which helps improve content consistency in subsequent batch generation.

[0048] In some embodiments of the present invention, step S102, where the master node determines the dependencies between atomic tasks, includes: Construct a directed acyclic graph (DAG) and use the DAG to label the plot continuity dependencies between scenes; Specifically, when the last frame of the previous shot in a continuous storyboard is used as the first frame of the next shot, a dependency relationship is marked between the video generation tasks corresponding to the two shots; no dependency relationship exists between the other atomic tasks.

[0049] This embodiment uses a Directed Acyclic Graph (DAG) to record and represent the pre- and post-constraints between tasks, in order to determine the dependencies between atomic tasks.

[0050] Specifically, the master node marks the dependencies in the DAG based on the plot continuity between scenes.

[0051] It's important to note that not all storyboards have dependencies. A dependency exists only when the last frame of one storyboard in a continuous narrative needs to be used as the first frame of the next storyboard; otherwise, all atomic tasks are independent and can be executed in parallel.

[0052] For example, in the first shot, the protagonist pushes open the door and steps out with their left foot. The second shot continues the scene after the protagonist's foot lands. At this point, the last frame of the first shot (the foot is suspended in the air) and the first frame of the second shot (the foot touches the ground) need to be continuous. Therefore, the video generation tasks of these two shots are dependent. There are no dependencies between image, audio, and compositing tasks within the same shot, or between image tasks, audio tasks, and compositing tasks in different shots.

[0053] In this way, this embodiment maximizes the number of tasks that can be executed in parallel while ensuring the continuity of the plot, and incorporates most tasks into the parallel execution queue.

[0054] In some embodiments of the present invention, such as Figure 3 As shown, step S103 distributes atomic tasks with no dependencies to multiple slave nodes in the distributed cluster in parallel via message queues, including: S301. Obtain the computing power utilization rate and computing power level of each slave node; S302. Calculate the scheduling cost of each atomic task relative to each slave node based on the task type weight, business priority, computing power utilization and computing power level of each slave node. S303. For each atomic task, select the slave node with the lowest scheduling cost as the target execution node for the atomic task, and distribute the atomic task to the target execution node.

[0055] Specifically, before executing the distribution, the master node first obtains the real-time computing power utilization and computing power level of each slave node.

[0056] The computing power utilization rate reflects the load of the current node's central processing unit (CPU) or graphics processing unit (GPU), with a value ranging from 0 to 1. This data is reported in real time by the acquisition agent built into the slave node. The computing power level represents the hardware performance of the node. For example, the computing power level of the A100 GPU is 10, the computing power level of the RTX 4090 GPU is 8, and the computing power level of a regular CPU is 1.

[0057] Then, the master node calculates a scheduling cost for each atomic task relative to each slave node based on the task type weight of the atomic task to be distributed, the business priority of the task, and the computing power utilization and computing power level of each slave node obtained earlier.

[0058] In some embodiments of the present invention, step S302, calculating the scheduling cost of each atomic task relative to each slave node, includes: The scheduling cost is calculated using the following formula. Cost ( i , j ):

[0059] in, T i The weights for the atomic task types are as follows: preferably, in this embodiment, the weight of the text-to-image task is 12, the weight of the text-to-video task is 15, the weight of the audio task is 1, and the weight of the synthesis task is 3.

[0060] in, P i This represents the business priority of the atomic task. For example, high priority is 1.0, medium priority is 0.6, and low priority is 0.3.

[0061] in, U j For the node j computing power utilization rate C j For the node j The computing power level, α and β For preset weighting coefficients, ε It is a very small positive number (with a value of 0.01) to prevent the denominator from being zero.

[0062] Preferably, in this embodiment α The value is 0.8. β The value is 0.2. It should be noted that this embodiment will α The weight is increased to 0.8 to ensure a reasonable allocation of computing resources; this is the core technology of this embodiment. β A weight of 0.2 is used to ensure business priority, which is the core of the business. The lower the scheduling cost, the more suitable the slave node is to execute this atomic task.

[0063] For each atomic task, the master node selects the slave node with the lowest scheduling cost as the target execution node and distributes the task there. This embodiment also sets a load threshold: when the computing power utilization of a slave node reaches a certain threshold... U j When the value is greater than or equal to 0.85, the node is determined to be in a high-load state, and the assignment of new tasks to it is suspended; when U j When the value is less than or equal to 0.4, the node is considered to be in a low-load state, and tasks are preferentially assigned to it. If the slave node remains in a high-load state, the system will automatically trigger the expansion mechanism to increase the number of slave nodes to share the pressure.

[0064] For example, an urgent, high-priority video generation task will be preferentially assigned to a slave node with high computing power and low current load; while a low-priority audio synthesis task may be assigned to a node with average computing power but also low load.

[0065] In some embodiments of the present invention, such as Figure 4 As shown, in step S105, the master node reads the execution results of each atomic task from the distributed storage system, merges them according to dependencies and the time sequence of the short drama script, and generates a short drama video, including: S401. Read the character feature vectors of the same character from the metadata corresponding to different scenes in the distributed storage system, and calculate the cosine similarity between the feature vectors of each character. S402. When the cosine similarity is less than or equal to the preset threshold, it is determined that there is an inconsistency in the character image of the corresponding storyboard, and the regeneration of the atomic task corresponding to the storyboard is automatically triggered. S403. When the cosine similarity of all segments is greater than the preset threshold, the execution results of each atomic task are read from the distributed storage system in the segment sequence. The execution results include the synthesized segments corresponding to each segment. S404. Combine the synthesized segments corresponding to each storyboard to generate a short drama video file and output it.

[0066] First, the master node reads the character feature vectors stored in the metadata corresponding to different scenes of the same character from the distributed storage system, and calculates the cosine similarity between each pair of these feature vectors.

[0067] It should be understood that the closer the cosine similarity is to 1, the more similar the two vectors are, which means that the character's image is more consistent across different scenes. The pre-set similarity threshold in this embodiment is 0.9.

[0068] When the calculated cosine similarity is less than or equal to 0.9, it is determined that there is an inconsistency in the character design of the corresponding storyboard. The system will automatically trigger the atomic tasks (including image, video, audio, and compositing tasks) corresponding to that storyboard to be regenerated. Conversely, when the cosine similarity of all storyboards is greater than 0.9, it means that the character design remains consistent throughout the entire series. At this time, the master node reads the complete storyboard fragments output by the compositing task (CLIP) corresponding to each storyboard from the distributed storage system in chronological order.

[0069] It should be noted that the output of the compositing task already includes the video footage, audio narration, background music, and subtitles for that storyboard, making it a ready-to-use audio-visual file. In other words, each storyboard corresponds to a pre-composited, independent segment.

[0070] Finally, the master node uses FFmpeg to seamlessly stitch together the composited segments corresponding to these storyboards, generating the final short drama video file. The entire episode and production report are then written to a designated directory in the Ceph distributed storage system. This embodiment employs a "verify first, then composite" strategy, avoiding invalid composites of substandard materials. Simultaneously, it achieves automated closed-loop correction of character consistency through cosine similarity threshold judgment.

[0071] In some embodiments of the present invention, such as Figure 5 As shown, the method for batch generation of this short drama includes: S501, The master node monitors the heartbeat signals of each slave node and the execution time of atomic tasks; S502. When the master node fails to receive a heartbeat signal from the slave node for a preset number of consecutive times, it determines that the slave node is faulty, reallocates the unfinished atomic tasks on the slave node to other slave nodes, and instructs the newly allocated slave nodes to read the checkpoint data of the atomic tasks from the distributed storage system to continue execution. S503. When the execution time of an atomic task exceeds the preset timeout threshold, the master node determines that the atomic task has failed and triggers automatic retry. If the number of retries exceeds the preset number of times threshold, it will perform downgrade generation or issue an alarm.

[0072] To ensure the stability of unattended production 24 / 7, this embodiment provides a fault tolerance mechanism. The fault tolerance and Checkpoint modules in the master node continuously monitor the heartbeat signals of each slave node and the execution time of each atomic task.

[0073] Specifically, the master node sends a heartbeat check every 5 seconds. If it does not receive a heartbeat response from a slave node for 3 consecutive times (i.e., 15 consecutive seconds), the master node determines that the slave node has failed.

[0074] At this point, the master node will reassign the unfinished atomic tasks on the failed node to other healthy slave nodes, and instruct the newly acquired slave node to read the checkpoint data of the task from the Ceph distributed storage system and continue execution from the breakpoint instead of starting from the beginning.

[0075] On the other hand, the master node also monitors the runtime of ongoing tasks. Different types of tasks have preset timeout thresholds; for example, text-to-image tasks have a maximum timeout of 10 minutes, text-to-video tasks have a maximum timeout of 20 minutes, and audio tasks have a maximum timeout of 8 minutes. Once the execution time of a task exceeds the corresponding timeout threshold, the master node determines that the task has failed and automatically triggers a retry mechanism.

[0076] The fault tolerance strategy in this embodiment is as follows: First, automatically retry twice; if it still fails after two retries, perform a downgraded generation (e.g., use a lower resolution or simplified version of the model); if the downgraded generation still fails, issue an alarm to the administrator.

[0077] In addition, this embodiment also uses Redis distributed locks combined with Ceph file atomic write mechanism to prevent the same task from being executed repeatedly and to prevent file corruption.

[0078] Through the multi-layered fault-tolerant design of the above-mentioned heartbeat detection, task migration, breakpoint resume, timeout retry, degradation alarm and distributed lock, this embodiment can effectively deal with various abnormal situations in the production process and ensure the stability of the batch generation of short dramas.

[0079] In some embodiments of the present invention, such as Figure 6 As shown, in image generation and / or video generation tasks, the consistency of character images in the generated content is constrained by character feature vectors. S601. The character feature vector is generated in the following way: the character image features are extracted using the CLIP or EVA-CLIP model, mapped to a preset dimension through a fully connected network, and ReLU is used as the activation function. S602. When performing the image generation task, the character feature vector is concatenated to the text embedding layer of the text-generated image model; S603. When executing a video generation task, read the key frames output by the image generation task from the distributed storage system as input frames.

[0080] This embodiment demonstrates how to constrain the consistency of character appearance in image generation and video generation tasks using character feature vectors. The method for generating character feature vectors has been described in detail in the preceding embodiments and will not be repeated here.

[0081] When performing image generation tasks, the slave nodes receive tasks through the Kafka message queue and read metadata, character feature vectors, and reference materials from the Ceph distributed storage.

[0082] This embodiment uses a text-based image model (such as the Flux.2 model) to perform the image generation task. The inference parameters are set as follows: resolution 1080×1920, sampling steps 20, CFG Scale value of 3.5, and Euler sampler. During the generation process, the 1024-dimensional character feature vector is concatenated to the text embedding layer of the Flux.2 model, which is equivalent to telling the model "the main character in the generated image must look like this", thereby locking in the character's appearance from the source.

[0083] It should be noted that the text embedding layer is responsible for converting text descriptions into numerical representations that the model can understand. The addition of character feature vectors is equivalent to adding visual constraints in addition to textual conditions. The generated images (storyboard sequence frames) are written to Ceph storage in real time.

[0084] In performing the video generation task, this embodiment employs a text-to-image video model. The model's input consists of two parts: first, keyframes read from Ceph storage and output by the Flux.2 image generation task, serving as the starting or reference frames; and second, the text description of the storyboard. Inference parameters are set as follows: segment length 1 to 5 seconds, frame rate 30fps, motion intensity 0.6, and resolution 1080×1920. The output is a storyboard video segment without audio, also written to Ceph storage.

[0085] In performing the audio generation task, this embodiment uses the Qwen3-TTS multi-voice speech synthesis model. Inference parameters are set as follows: sampling rate 44.1kHz, bit rate 320kbps, speech rate 1.0, and corresponding emotions are matched according to the character. During audio mixing, the dialogue volume accounts for 70%, and the background music volume accounts for 30%, with the duration strictly aligned with the scene sequence. The output is an MP3 audio file, written to Ceph storage.

[0086] When performing the compositing task, the video clips generated by the Wan model and the audio files generated by Qwen3-TTS are read from Ceph storage, aligned and synthesized using FFmpeg, and subtitles are generated at the same time. Finally, the complete clip of the storyboard is output to Ceph storage.

[0087] Through multiple mechanisms, including "vector constraints during image generation, image output as input during video generation, and strict alignment of audio and video," this embodiment ensures that the same character maintains a stable visual and auditory image in different storyboards and even different episodes.

[0088] To better implement the short drama production batch generation method in this invention embodiment, based on the short drama production batch generation method, correspondingly, as follows: Figure 7 As shown, this embodiment of the invention also provides a device for batch generation of short drama works. The device 700 for batch generation of short drama works includes: Data acquisition module 701 is used to acquire the script of the short drama to be generated; The task decomposition module 702 is used to decompose the short drama script into multiple atomic tasks and determine the dependencies between each atomic task. The atomic tasks include image generation tasks, video generation tasks, audio generation tasks, and compositing tasks. The parallel distribution module 703 is used to distribute atomic tasks that do not have dependencies to multiple slave nodes in the distributed generation system in parallel through a message queue, based on the dependencies. The parallel generation module 704 is used to execute the received atomic tasks in parallel and write the intermediate data generated during execution as the execution result into the distributed storage system. The short drama synthesis module 705 is used to read the execution results of each atomic task from the distributed storage system, merge them according to the dependencies and the time sequence of the short drama script, and generate short drama videos.

[0089] The short drama production batch generation device 700 provided in the above embodiments can realize the technical solutions described in the above short drama production batch generation method embodiments. The specific implementation principles of each module or unit can be found in the corresponding content in the above short drama production batch generation method embodiments, and will not be repeated here.

[0090] like Figure 8 As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0091] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as the short drama production batch generation method of the present invention.

[0092] In some embodiments, processor 801 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 801 may be local or remote. In some embodiments, processor 801 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0093] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0094] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0095] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display information from electronic device 800 and to display a visual user interface. Components 801-803 of electronic device 800 communicate with each other via a system bus.

[0096] In one embodiment, when the processor 801 executes the short drama production batch generation program in the memory 802, the following steps can be implemented: The master node retrieves the script for the short drama to be generated; The master node breaks down the short drama script into multiple atomic tasks and determines the dependencies between each atomic task. The atomic tasks include image generation, video generation, audio generation, and compositing tasks. Based on dependencies, the master node distributes atomic tasks that do not have dependencies to multiple slave nodes in the distributed generation system in parallel through a message queue; Each slave node executes the received atomic tasks in parallel and writes the intermediate data generated during execution into the distributed storage system as the execution result; The master node reads the execution results of each atomic task from the distributed storage system, merges them according to dependencies and the time sequence of the short drama script, and generates the short drama video.

[0097] It should be understood that when the processor 801 executes the short drama production batch generation program in the memory 802, in addition to the functions mentioned above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0098] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0099] Accordingly, this application also provides a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the short drama production batch generation method provided in the above-described method embodiments.

[0100] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0101] The above provides a detailed description of the method, apparatus, and electronic device for batch generation of short drama works provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for batch generation of short drama works, characterized in that, Applied to a distributed generation system, the distributed generation system including a master node and at least two slave nodes, the method for batch generation of short drama works includes: The master node obtains the script for the short drama to be generated; The master node breaks down the short drama script into multiple atomic tasks and determines the dependencies between each atomic task. The atomic tasks include image generation, video generation, audio generation, and synthesis tasks. The master node distributes atomic tasks that do not have dependencies to multiple slave nodes in the distributed generation system in parallel through a message queue, based on the dependency relationship. Each slave node executes the received atomic tasks in parallel and writes the intermediate data generated during execution into the distributed storage system as the execution result; The master node reads the execution results of each atomic task from the distributed storage system, merges them according to the dependencies and the time sequence of the short drama script, and generates a short drama video.

2. The method for batch generation of short drama works according to claim 1, characterized in that, The master node breaks down the short drama script into multiple atomic tasks, including: The large language model is used to extract character information, scene information, shot information, dialogue text, and shot duration information from the short drama script; The tasks are broken down according to the hierarchy of episodes and storyboards, and each storyboard corresponds to four atomic tasks, including an image generation task, a video generation task, an audio generation task, and a compositing task. Metadata is generated for each storyboard, including character information, scene information, shot information, dialogue text, and storyboard duration information. The metadata is then written into the distributed storage system, wherein each atomic task corresponding to the same storyboard is associated with the metadata corresponding to that storyboard.

3. The method for batch generation of short drama works according to claim 2, characterized in that, The master node determines the dependencies between atomic tasks, including: Construct a directed acyclic graph, and label the plot continuity dependencies between scenes based on the directed acyclic graph; Specifically, when the last frame of the previous shot in a continuous storyboard is used as the first frame of the next shot, a dependency relationship is marked between the video generation tasks corresponding to the two shots; no dependency relationship exists between the other atomic tasks.

4. The method for batch generation of short drama works according to claim 1, characterized in that, The method of distributing atomic tasks with no dependencies to multiple slave nodes in a distributed cluster in parallel via message queues includes: Obtain the computing power utilization and computing power level of each slave node; Based on the task type weight and business priority of the atomic task, as well as the computing power utilization and computing power level of each slave node, calculate the scheduling cost of each atomic task relative to each slave node. For each atomic task, the slave node with the lowest scheduling cost is selected as the target execution node for that atomic task, and the atomic task is distributed to the target execution node.

5. The method for batch generation of short drama works according to claim 4, characterized in that, The calculation of the scheduling cost of each atomic task relative to each slave node includes: The scheduling cost is calculated using the following formula. Cost ( i , j ): in, T i The task type weight for atomic tasks. P i The business priority of atomic tasks, U j For the node j computing power utilization rate C j For the node j The computing power level, α and β For preset weighting coefficients, ε It is a very small positive number.

6. The method for batch generation of short drama works according to claim 1, characterized in that, The master node reads the execution results of each atomic task from the distributed storage system, merges them according to the dependencies and the time sequence of the short drama script, and generates a short drama video, including: Read the character feature vectors of the same character from the metadata corresponding to different scenes in the distributed storage system, and calculate the cosine similarity between the feature vectors of each character; When the cosine similarity is less than or equal to a preset threshold, it is determined that there is an inconsistency in the character image of the corresponding storyboard, and the regeneration of the atomic task corresponding to the storyboard is automatically triggered. When the cosine similarity of all scenes is greater than the preset threshold, the execution results of each atomic task are read from the distributed storage system according to the scene sequence. The execution results include the synthesized segments corresponding to each scene. The pre-composite segments corresponding to each scene are spliced ​​together to generate a short drama video file and output it.

7. The method for batch generation of short drama works according to any one of claims 1 to 6, characterized in that, The method for batch generation of short drama works includes: The master node monitors the heartbeat signals of each slave node and the execution time of atomic tasks; When the master node fails to receive a heartbeat signal from the slave node for a preset number of consecutive times, it determines that the slave node is faulty, reallocates the unfinished atomic tasks on the slave node to other slave nodes, and instructs the newly allocated slave nodes to read the checkpoint data of the atomic tasks from the distributed storage system to continue execution. When the execution time of an atomic task exceeds a preset timeout threshold, the master node determines that the atomic task has failed and triggers automatic retry. If the number of retries exceeds a preset threshold, it will perform a downgrade or issue an alarm.

8. The method for batch generation of short drama works according to any one of claims 1 to 6, characterized in that, In the image generation task and / or video generation task, the consistency of the character image in the generated content is constrained by the character feature vector; The character feature vector is generated in the following way: the character image features are extracted using the CLIP or EVA-CLIP model, mapped to a preset dimension through a fully connected network, and ReLU is used as the activation function. When performing the image generation task, the character feature vector is concatenated to the text embedding layer of the text-generated image model; When performing the video generation task, the keyframes output by the image generation task are read from the distributed storage system as input frames.

9. A device for mass-producing short drama works, characterized in that, include: The data acquisition module is used to acquire the script of the short drama to be generated; The task decomposition module is used to decompose the short drama script into multiple atomic tasks and determine the dependencies between each atomic task. The atomic tasks include image generation tasks, video generation tasks, audio generation tasks, and synthesis tasks. The parallel distribution module is used to distribute atomic tasks that do not have dependencies to multiple slave nodes in the distributed generation system in parallel through a message queue, based on the dependencies. The parallel generation module is used to execute the received atomic tasks in parallel and write the intermediate data generated during execution into the distributed storage system as the execution result. The short drama synthesis module is used to read the execution results of each atomic task from the distributed storage system, merge them according to the dependencies and the time sequence of the short drama script, and generate a short drama video.

10. An electronic device, characterized in that, Including memory and processor, among which, The memory is used to store programs; The processor, coupled to the memory, is used to execute the program stored in the memory to implement the steps in the method for batch generation of short drama works according to any one of claims 1 to 8.

Images