Video generation method, device and system based on multi-agent cooperation

Abstract

The present disclosure provides a multi-agent cooperation-based video generation method, device and system, relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, large model, agent, artificial intelligence generated content, video generation and the like, and is applied to the application scenarios of smart education, video production, animation production and the like. The specific implementation scheme is: based on demand information, determining description information of a plurality of sub-tasks for generating a video; based on the description information of each sub-task, controlling a corresponding sub-task agent to determine the execution result of each sub-task; performing consistency verification on the execution result of each sub-task agent; in response to the execution result of each sub-task agent being successfully verified, based on the execution result of each sub-task agent, controlling a video generation agent to generate a video.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to the fields of deep learning, large models, intelligent agents, AI-generated content, and video generation, specifically to a video generation method, apparatus, and system based on multi-agent collaboration. Background Technology

[0002] Currently, the industry's AIGC (AI Generated Content) video generation method still mainly relies on a step-by-step workflow of "manually connecting independent modules," and each step of this workflow cannot be separated from human review, correction, and coordination.

[0003] Because existing AIGC models (including large language models and image / video generation models) generally suffer from the illusion problem, the generated results often deviate from the preset requirements (such as irrelevant plots in the script, inconsistencies in character portrayals, and asynchrony between video clips and dialogue). This forces operators to repeatedly debug prompts for each stage of the output (such as supplementing detailed constraints, correcting logical loopholes, and clarifying style boundaries). Some complex requirements even require dozens of trials to achieve the desired effect. This model not only significantly increases labor and time costs but also severely limits production efficiency. The actual production cycle of a single video often takes several hours, and the final effect is highly dependent on the operator's experience and professional judgment in debugging prompts, making it difficult to achieve standardized and large-scale output. Summary of the Invention

[0004] This disclosure provides a video generation method, apparatus, system, device, storage medium, and computer program product based on multi-agent collaboration.

[0005] According to a first aspect of this disclosure, a video generation method based on multi-agent collaboration is provided, comprising: determining description information of multiple sub-tasks for generating a video based on demand information; controlling corresponding sub-task agents based on the description information of each sub-task to determine the execution result of each sub-task; performing consistency verification on the execution results of each sub-task agent; and, in response to the successful verification of the execution results of each sub-task agent, controlling a video generation agent to generate a video based on the execution results of each sub-task agent.

[0006] According to a second aspect of this disclosure, a video generation apparatus based on multi-agent collaboration is provided, comprising: an allocation unit configured to determine description information of multiple sub-tasks for generating video based on demand information; a control unit configured to control corresponding sub-task agents based on the description information of each sub-task and determine the execution result of each sub-task; a verification unit configured to perform consistency verification on the execution results of each sub-task agent; and a generation unit configured to, in response to the successful verification of the execution results of each sub-task agent, control a video generation agent to generate video based on the execution results of each sub-task agent.

[0007] According to a third aspect of this disclosure, a video generation system based on multi-agent collaboration is provided, comprising: a central control agent configured to execute the method of any one of the first aspects; a subtask agent configured to determine the execution result of a subtask based on description information of the subtask determined by the central control agent; and a video generation agent configured to generate a video based on the execution result of the subtask agent.

[0008] According to a fourth aspect of this disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method described in any one of the first aspects.

[0009] According to a fifth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in any one of the first aspects.

[0010] According to a sixth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in any one of the first aspects.

[0011] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0012] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 This is an architecture diagram of a video generation system based on multi-agent collaboration according to the present disclosure; Figure 2 This is a flowchart of an embodiment of the video generation method based on multi-agent cooperation according to the present disclosure; Figures 3a-3b The figure is a schematic diagram of an application scenario of the video generation method based on multi-agent cooperation according to this disclosure; Figure 4 This is a flowchart of yet another embodiment of the video generation method based on multi-agent cooperation according to the present disclosure; Figure 5 This is a schematic diagram of a structure of an embodiment of a video generation apparatus based on multi-agent collaboration according to the present disclosure; Figure 6 This is a schematic diagram of the structure of a computer system suitable for implementing embodiments of the present disclosure. Detailed Implementation

[0013] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0014] Figure 1 An exemplary system architecture 100 is shown, to which embodiments of the multi-agent collaborative video generation method or multi-agent collaborative video generation apparatus of this disclosure can be applied.

[0015] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0016] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as video generation applications, 3D video players, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0017] Terminal devices 101, 102, and 103 can be either hardware or software. When terminal devices 101, 102, and 103 are hardware, they can be various electronic devices with displays that support 3D video playback, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc. When terminal devices 101, 102, and 103 are software, they can be installed in the aforementioned electronic devices. They can be implemented as multiple software programs or software modules (e.g., to provide distributed services) or as a single software program or software module. No specific limitations are imposed here.

[0018] Server 105 can be a server providing various services. Taking video generation services as an example, the server can install a central control agent, multiple sub-task agents, and a video generation agent. The sub-task agents may include: an outline / script agent, a character / scene agent, a character reference image agent, and a storyboard agent (which may include a storyboard design agent, a storyboard image generation agent, and a storyboard quality inspection agent). The central control agent breaks down the video task into sub-tasks based on the requirements, assigns them to the sub-task agents for execution, and performs consistency checks on the execution results of each sub-task agent. If the execution results of each sub-task agent are successfully checked, the central control agent controls the video generation agent to generate the video based on the execution results of each sub-task agent.

[0019] It's important to note that a server can be either hardware or software. When a server is hardware, it can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. When a server is software, it can be implemented as multiple software programs or software modules (e.g., multiple software programs or software modules used to provide distributed services), or as a single software program or software module. No specific limitations are made here. A server can also be a server for a distributed system, or a server integrated with blockchain technology. A server can also be a cloud server, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0020] It should be noted that the video generation method based on multi-agent collaboration provided in the embodiments of this disclosure is generally executed by server 105, and correspondingly, the video generation device based on multi-agent collaboration is generally set in server 105.

[0021] It should be understood that Figure 1The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0022] Continue to refer to Figure 2 The diagram illustrates a flow 200 of an embodiment of a video generation method based on multi-agent cooperation according to the present disclosure. This video generation method based on multi-agent cooperation includes the following steps: Step 201: Based on the requirements information, determine the description information of multiple sub-tasks used to generate the video; In this embodiment, the central control agent, acting as the "central brain" of the entire multi-agent system, assumes the core responsibility of overall coordination and closed-loop management. It first receives the core user input, which may include basic information such as product selling points, target audience, and narrative style. Through high-level CoT (Chain of Thought), it performs global reasoning to clarify the core guiding principles for video generation. For example, for e-commerce short dramas, it determines the core narrative framework and production standards as follows: "Identification of user needs gaps and scenario-based presentation - Matching of product core features with needs and content integration - Guidance of target operations and triggering of conversion paths." Subsequently, the central control agent breaks down the overall task into sub-modules such as outline generation, character design, and storyboard creation, and determines the descriptive information for multiple sub-tasks used to generate the video. These sub-tasks may include: generating outline scripts, determining character settings and scenes, generating character reference images, and generating storyboards.

[0023] Step 202: Based on the description information of each subtask, control the corresponding subtask agent to determine the execution result of each subtask; In this embodiment, each subtask is precisely assigned to a corresponding functional agent, which then executes the subtask and synchronously transmits the collaborative logic of each stage. The central control agent can obtain the execution results of all subtasks. The execution results of each subtask may include: outline script, character design scenes, character reference images, storyboards, etc.

[0024] Step 203: Perform consistency verification on the execution results of each subtask agent; In this embodiment, throughout the entire process, the central control agent continuously verifies the consistency of the outputs of each agent—including the matching degree between the plot logic and product requirements, the adaptability of character design and scene, and the coherence between storyboards and outlines. At the same time, as a feedback hub, when problems are detected in subsequent stages, adjustment instructions are triggered in a timely manner, linking relevant agents to carry out iterative optimization, ensuring that the entire production process always revolves around the goal of "high quality, high conversion, and strong consistency," and avoiding disconnection of stages or deviation from requirements.

[0025] Step 204: In response to the successful verification of the execution results of each subtask agent, control the video generation agent to generate the video based on the execution results of each subtask agent.

[0026] In this embodiment, the video generation agent, acting as the "terminal execution link" in the entire process, is responsible for converting qualified storyboard images into complete AIGC short drama products, achieving automated generation from "first frame to continuous video." It uses the quality-approved storyboard images as the first frame, combines the clearly defined camera movement instructions (such as push-pull, transitions, and shot changes) from the storyboard design agent, and calls a video generation model adapted for e-commerce short dramas to generate continuous and smooth video segments. For audio matching, it automatically connects to speech synthesis technology to generate dialogue that matches the characters' tone, and adds background sound effects that fit the plot style, ultimately outputting a complete video containing visuals, audio, and sound effects. This eliminates the need for post-production splicing, improving video generation efficiency and quality while reducing video production costs. The video generation agent can employ a general video generation model.

[0027] The method provided in the above embodiments of this disclosure, by constructing a multi-agent system of "central control agent + multiple sub-task agents", deeply integrates storyboard quality inspection, genetic algorithm prompt word optimization, and hierarchical CoT (Chain of Thought) reasoning mechanism into the entire process, completely replacing the serial mode, greatly reducing core problems such as model illusion, lack of consistency, and low efficiency, improving video quality and video generation efficiency, and reducing video production costs.

[0028] The system establishes a collaborative architecture of "central control agent + six functional agents," with each agent achieving closed-loop collaboration through a hierarchical CoT mechanism of "global reasoning - local execution - feedback correction." Existing technologies require constantly changing storyboard scenes to address scene consistency issues. This solution, while ensuring consistency of characters appearing in different scenes within the video, significantly improves the accuracy of storyboarding by designing a scene consistency agent and incorporating genetic algorithms and quality control modules, thereby enhancing video quality.

[0029] In some optional implementations of this embodiment, the method further includes: in response to the failure of execution result verification of any subtask agent, adjusting the prompt word of the subtask agent that failed the verification; based on the adjusted prompt word, controlling the subtask agent that failed the verification to re-execute the corresponding subtask and update the execution result; performing consistency verification on the updated execution result; in response to the successful consistency verification of the updated execution result, controlling the video generation agent to generate a video based on the successful consistency verification execution result.

[0030] If the verification fails, the prompt words can be regenerated based on the reason for the failure, and the relevant intelligent agents can be linked to carry out iterative optimization to ensure that the entire production process always maintains consistency with the demand information and avoids the disconnect between links or deviation from the demand.

[0031] In some optional implementations of this embodiment, based on the requirement information, the description information of multiple sub-tasks for generating video is determined, including: based on the requirement information, high-level global reasoning is performed through a hierarchical thinking chain mechanism to determine the global framework of the video generation task; based on the global framework, the video generation task is decomposed into multiple sub-tasks, and the description information of each sub-task is determined.

[0032] By using high-level, holistic reasoning to directly anchor the core requirements, subtasks can be prevented from deviating from the overall goal. First, clearly define the core theme, style, narrative logic, and other global elements of the video to create unified constraints. This avoids content fragmentation caused by prioritizing local details, ensuring the integrity and consistency of the generated video.

[0033] In some optional implementations of this embodiment, multiple subtasks include: an outline script subtask; and, based on the description information of each subtask, controlling the corresponding subtask agent to determine the execution result of each subtask, including: based on the description information of the outline script subtask, controlling the outline script agent to determine the outline of the video through a high-level thought chain; and based on the outline, controlling the outline script agent to determine the plot and dialogue through a low-level thought chain.

[0034] The outline scripting agent focuses on the "narrative core" of the video, supported by a hierarchical CoT mechanism. It specifically addresses issues in existing technologies such as broken plot logic, awkward product placement, and irrelevant content generation due to model illusions. Within the global framework defined by the central control agent, after inputting product information, it first uses high-level CoT reasoning to determine the short-duration (30-60 seconds) narrative rhythm, establishing a fixed structure of "5 seconds to capture the pain point, 10 seconds to create conflict, 10 seconds for strong product placement, and 5 seconds to drive conversion." Then, it fills in detailed content through low-level CoT: it reverse-engineers the core pain points of the target user based on the product's selling points (e.g., the moisturizing selling point of skincare products corresponds to the pain point of "dry, flaky skin in autumn and winter"), designing natural dialogue that fits the character's identity. This ensures the dialogue aligns with the character's personality while subtly conveying the product's advantages, avoiding blatant advertising. Simultaneously, it synchronizes its plot reasoning process with subsequent agents, providing logical basis for character design, scene composition, and shot composition, reducing the generation of irrelevant plot (model illusions) from the source.

[0035] The outline script agent can use a variety of models to generate text content and generate outline scripts in parallel. The best plot and dialogue are selected by the overall control agent.

[0036] In some optional implementations of this embodiment, multiple subtasks include: character setting and scene subtasks; and based on the description information of each subtask, controlling the corresponding subtask intelligent agent to determine the execution result of each subtask, including: based on the plot and the plot's reasoning process information, controlling the character setting and scene intelligent agent to determine the character setting description information and scene description information through a hierarchical thinking chain mechanism, wherein the scene description information includes the spatial coordinate system position of the character and / or the spatial coordinate system position of the object.

[0037] The character design and scene intelligence agent is responsible for constructing the "visual and character foundation" of the video. Through adaptive reasoning using a hierarchical CoT mechanism, it ensures a high degree of consistency between character design, scene, and plot logic. It receives the core plot information (such as "fitness scene" or "best friend skincare scene") from the outline script intelligence agent. Combining this with audience characteristics from user needs, it first infers and determines the core attributes of the character: including age, occupation, personality (such as "25-year-old white-collar worker, capable personality, focusing on efficient skincare"), and clothing style (such as "simple commuter attire, suitable for the workplace scene"), ensuring that the character image can resonate with the target audience. Then, it refines the design of the plot scene, clarifying the scene type (lifestyle / workplace / promotion), core elements (such as a bedroom scene needing to include a dressing table, skincare products, and soft lighting), and lighting style (such as using natural light in a daytime scene to create a sense of realism). At the same time, it establishes the scene's "spatial coordinate system" (such as the sofa on the left and the coffee table in the middle), providing a spatial consistency basis for subsequent storyboard generation. The character and scene descriptions it outputs will serve as the core basis for subsequent reference image generation and storyboard design, avoiding problems such as "character design being disconnected from the plot" and "scenes not matching the selling points".

[0038] The character-scene intelligent agent can select from a variety of models for generating text content, generate character description information and scene description information in parallel, and select the best character description information and scene description information through evaluation by the overall control intelligent agent.

[0039] In some optional implementations of this embodiment, multiple subtasks include: a character reference image subtask; and controlling the corresponding subtask agent based on the description information of each subtask to determine the execution result of each subtask, including: controlling the character reference image agent based on the character description information to determine the character reference image.

[0040] The core task of the character reference image agent is to generate standardized, highly recognizable, and reusable character reference images. This provides a unified anchor point for character features in subsequent storyboard and video generation, solving the character consistency problem from the source. The final generated reference images include multiple angles such as front, side, and half-body views, clearly marking core features such as facial contours, hairstyles, and clothing details. This ensures that all characters in subsequent storyboards are based on this reference image, avoiding abrupt changes in appearance.

[0041] The intelligent agent for the reference image of a person can choose a model used to generate the image.

[0042] In some optional implementations of this embodiment, controlling the character reference image agent based on character description information to determine the character reference image includes: controlling the character reference image agent based on character description information to determine multiple initial prompt words; determining the initial prompt word to be optimized from the multiple initial prompt words; controlling the character reference image agent to optimize the initial prompt word to be optimized; and controlling the character reference image agent to determine the character reference image based on the initial prompt word that does not need optimization and the optimized initial prompt word.

[0043] Based on the detailed character descriptions output by the character-setting scene agent, a genetic algorithm is used to automatically optimize the prompts. After 3-5 rounds of optimization, the optimal prompts are output. The character reference image agent then generates an accurate character reference image based on the optimal prompts, making the character more vivid and three-dimensional, thereby improving the quality of the final video.

[0044] In some optional implementations of this embodiment, controlling the character reference image agent to optimize the initial prompt words to be optimized includes: controlling the character reference image agent to cross-fuse the initial prompt words that do not need optimization to determine the core feature words; controlling the character reference image agent to mutate and optimize the initial prompt words to be optimized to determine the mutated words; controlling the character reference image agent to determine the optimized initial prompt words based on the core feature words and mutated words through a genetic algorithm.

[0045] For example, firstly, a batch of initial prompt words containing core keywords (such as "25-year-old female, round face, shoulder-length hair, white commuter shirt"), style constraints, and detailed supplements are generated; then, the overall control agent combines the "match degree with the character description" to perform fitness scoring and select high-scoring prompt words; through an iterative process of "cross fusion (extracting the core features of high-scoring prompt words) + mutation optimization (supplementing facial details, clothing texture, posture angles, and other constraints)," after 3-5 rounds of optimization, the optimal prompt words are output.

[0046] In some optional implementations of this embodiment, multiple subtasks include: storyboarding task; and controlling the corresponding subtask agent based on the description information of each subtask to determine the execution result of each subtask, including: controlling the storyboard agent to generate storyboard based on the outline, character description information, scene description information and character reference image.

[0047] Storyboards can be generated directly by a single agent, or the storyboard task can be broken down into multiple sub-tasks to ultimately generate storyboards.

[0048] The generated storyboards can be quality checked. If the character's appearance (facial features, hairstyle, clothing) in the storyboard is not completely consistent with the reference image, or if the scene elements (prop positions, lighting effects) do not conform to the spatial rules established by the character and scene AI, prompts can be generated based on the inconsistencies to control the storyboard AI and regenerate the storyboards. This provides a high-quality, highly consistent first frame foundation for subsequent video generation.

[0049] In some optional implementations of this embodiment, multiple subtasks include a storyboard design subtask and a storyboard generation subtask; and based on the description information of each subtask, the corresponding subtask agent is controlled to determine the execution result of each subtask, including: based on plot, dialogue, character description information, and scene description information, controlling the storyboard design agent to determine the total number of storyboards and the overall rhythm through a high-level thought chain; based on the total number of storyboards and the overall rhythm, controlling the storyboard design agent to determine the storyboard design scheme through a low-level thought chain; and based on the storyboard design scheme and character reference images, controlling the storyboard generation agent to generate storyboards.

[0050] The storyboard design agent is a key link connecting "narrative text" and "visual presentation". Through the hierarchical CoT mechanism of lens language reasoning, it transforms the outline, character design and scene into executable storyboard instructions. It first receives the plot and dialogue from the outline script agent and the character and scene descriptions from the character and scene agents. Through high-level CoT inference, it determines the total number of scenes and the overall rhythm (e.g., 6-8 scenes for a 30-second short drama to ensure information increment in each frame). Then, through low-level CoT, it refines the design of each scene: clarifying the shot type (long shot / medium shot / close shot / extreme close-up) - such as "use a medium shot when the character complains about pain points to highlight emotions; use an extreme close-up when showcasing products to highlight details", camera movement (push-pull / pan / fixed) - such as "use a push-pull shot when the character hands over a product to focus on the product; use fade-in / fade-out when switching scenes to improve smoothness", character actions (e.g., "character A frowns and touches their cheek to show dry skin; character B reaches out and hands over skincare products, the action is natural"), and dialogue matching (ensuring that the on-screen actions and dialogue are synchronized, such as "when saying 'Try this,' the character's hand gestures towards the product"). Its output storyboard design scheme (including the shot size, camera movement, action, and dialogue of each storyboard) will directly guide the generation of subsequent storyboard diagrams, ensuring that the camera language conforms to the logic of professional live-action shooting and avoiding problems such as chaotic camera movement, stiff action, and disconnect between the visuals and the dialogue.

[0051] The storyboard design agent can use the same model as the outline generation agent.

[0052] The storyboard generation agent is responsible for converting storyboard design instructions into high-quality, visual storyboards (i.e., the first frame of the video). It relies heavily on genetically optimized prompts and previously generated character reference images to ensure consistency and accuracy. It receives detailed instructions from the storyboard design agent (e.g., "Medium shot, Character A is next to a dressing table in a bedroom, frowning and touching their cheek; background is a white dressing table and skincare products") and a character reference image generated by the character reference image agent. Then, based on the prompts, it generates storyboards, ensuring that the character's appearance (facial features, hairstyle, clothing) in each storyboard is completely consistent with the reference image, scene elements (prop positions, lighting effects) conform to the spatial rules established by the character and scene design agent, and actions and postures highly match the storyboard design instructions. This provides a high-quality, highly consistent first frame foundation for subsequent video generation.

[0053] The storyboard generation agent can use a model designed for generating images.

[0054] In some optional implementations of this embodiment, the process of controlling a storyboard generation agent to generate storyboards based on storyboard design schemes and character reference images includes: controlling a storyboard generation agent to generate prompts based on storyboard design schemes and character reference images; controlling the storyboard generation agent to optimize the prompts based on a genetic algorithm to determine optimized prompts; and controlling the storyboard generation agent to generate storyboards based on optimized prompts and character reference images.

[0055] The prompts can be further optimized using a genetic algorithm – supplementing "character traits and references" Figure 1 By applying constraints such as "scene elements conform to the spatial coordinate system" and "lighting and shadow style matches the scene description," accurate prompts are generated to adapt to the video generation model. This improves the accuracy of the prompts, thereby improving the accuracy of the storyboard.

[0056] In some optional implementations of this embodiment, multiple subtasks include a storyboard quality inspection subtask; and based on the description information of each subtask, controlling the corresponding subtask agent to determine the execution result of each subtask, including: controlling the storyboard quality inspection agent to perform quality inspection on the storyboard; in response to detecting an unqualified storyboard, controlling the storyboard generation agent based on a genetic algorithm to re-optimize the prompt words and determine new prompt words; and based on the new prompt words and character reference images, controlling the storyboard generation agent to generate storyboards.

[0057] The storyboard quality control AI is a quality control module that ensures content quality. Through a multi-dimensional verification mechanism, it intercepts issues such as inconsistencies in characters, scene contradictions, and logical breaks at the source. If a substandard storyboard is detected, it will be automatically fed back to the storyboard generation AI, triggering a genetic algorithm to re-optimize the prompts.

[0058] The storyboard quality inspection intelligent agent can adopt a multimodal model.

[0059] In some optional implementations of this embodiment, multiple subtasks include a storyboard quality inspection subtask; and based on the description information of each subtask, controlling the corresponding subtask intelligent agent to determine the execution result of each subtask, including: controlling the storyboard quality inspection intelligent agent to perform quality inspection on the storyboard; in response to detecting an unqualified storyboard, controlling the storyboard design intelligent agent to redetermine the storyboard design scheme; and based on the redetermined storyboard design scheme and character reference images, controlling the storyboard generation intelligent agent to generate the storyboard.

[0060] If an unqualified storyboard is detected, feedback is sent to the storyboard design agent to adjust the storyboard until a qualified storyboard is generated. By improving the accuracy of the storyboard design scheme, the accuracy of the storyboard is improved.

[0061] In some optional implementations of this embodiment, the video generation agent is controlled to generate video based on the execution results of each subtask agent, including: controlling the video generation agent to generate dialogue based on lines and character description information; controlling the video generation agent to generate video footage based on storyboards; controlling the video generation agent to determine background sound effects based on the plot; and controlling the video generation agent to synthesize video based on dialogue, video footage, and background sound effects.

[0062] The video generation agent, acting as the "final execution link" in the entire process, is responsible for transforming qualified storyboard images into complete AIGC short drama products, achieving automated generation from "first frame to continuous video." It uses the storyboard images approved by the storyboard quality control agent as the first frame, combined with the camera movement instructions (such as push-pull, transitions, and shot changes) clearly defined by the storyboard design agent, and calls upon a video generation model adapted for e-commerce short dramas to generate continuous and smooth video clips. For audio matching, it automatically connects to speech synthesis technology to generate dialogue that matches the characters' tone, paired with background sound effects adapted to the plot style, ultimately outputting a complete video containing visuals, audio, and sound effects, eliminating the need for post-production splicing and achieving end-to-end automation from input requirements to output.

[0063] In some optional implementations of this embodiment, consistency verification of the execution results of each subtask agent includes: verifying the consistency between the execution results of each subtask agent; and / or verifying the consistency between the requirement information and the execution results of each subtask agent.

[0064] The consistency between the execution result of each subtask agent and the requirement information can be verified. If the execution result of any subtask fails to be verified, the prompt words of the agent associated with that subtask can be readjusted and the subtask can be re-executed.

[0065] The consistency of the execution results of any two subtask agents can be verified. If the verification fails, the prompts of the agents associated with the two subtasks can be readjusted and the subtasks can be re-executed.

[0066] No additional verification is required, yet the logic of the plot, the rationality of the integration of core product features, and the fluency of the dialogue can still be guaranteed, avoiding the generation of AI-generated illusions.

[0067] In some optional implementations of this embodiment, based on the description information of each subtask, the corresponding subtask agent is controlled to determine the execution result of each subtask, including: for the target subtask, based on the description information of the target subtask, multiple large language models are run in parallel to determine multiple execution results of the target subtask; the best execution result is determined from the multiple execution results of the target subtask as the execution result of the target subtask.

[0068] The target subtask can be of any type. When computing power is limited, the target subtask can also be a subtask whose execution result is text, such as an outline script subtask, a character / scene design subtask, or a storyboard design subtask. The character reference image generation subtask and storyboard generation subtask, however, use only one model. This improves the accuracy of the storyboards, reduces data processing time, and increases user satisfaction.

[0069] See also Figure 3a and Figure 3b , Figure 3a and Figure 3b This is a schematic diagram illustrating the generation of two-person AIGC short dramas of different styles based on the multi-agent collaboration video generation method of this embodiment. Figure 3a It is an output of scene design. Figure 3b These are storyboard images from a short drama.

[0070] Further reference Figure 4 This illustrates a flow 400 of another embodiment of a video generation method based on multi-agent cooperation. The flow 400 of this multi-agent cooperation-based video generation method includes the following steps: Step 401: Convert the audio in the promotional video into text content, and determine the demand information based on the text content.

[0071] In this embodiment, demand information can be extracted from promotional videos. The audio in the promotional video is converted into text content using automatic speech recognition technology. Then, the semantics of the text content are identified using a large language model to determine the demand information. For example, if the dialogue in the promotional video involves promoting products, the demand information includes product selling points, target audience, and plot style.

[0072] Step 402: Based on the requirements information, determine the description information of multiple sub-tasks used to generate the video; Step 403: Based on the description information of each subtask, control the corresponding subtask agent to determine the execution result of each subtask; Step 404: Perform consistency verification on the execution results of each subtask agent; Step 405: In response to the successful verification of the execution results of each subtask agent, the video generation agent is controlled to generate the video based on the execution results of each subtask agent.

[0073] Steps 402-405 are basically the same as steps 201-204, so they will not be described again.

[0074] Step 406: Use the generated video as the intro and splice it with the promotional video to generate the target video.

[0075] In this embodiment, two videos are spliced ​​together, with the video generated using the method of this application serving as the intro to the promotional video. The video generated using the method of this application has the following characteristics: smooth dialogue, logical plot, and seamless character movements and camera transitions, avoiding common problems in existing technologies such as stiff movements, stuttering, and mismatch between scenes and characters.

[0076] Further reference Figure 5 As an implementation of the methods shown in the above figures, this disclosure provides an embodiment of a video generation device based on multi-agent collaboration, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0077] like Figure 5 As shown, the video generation device 500 based on multi-agent collaboration in this embodiment includes: an allocation unit 501, a control unit 502, a verification unit 503, and a generation unit 504. The allocation unit 501 is configured to determine the description information of multiple sub-tasks for video generation based on demand information; the control unit 502 is configured to control the corresponding sub-task agents based on the description information of each sub-task to determine the execution result of each sub-task; the verification unit 503 is configured to perform consistency verification on the execution results of each sub-task agent; and the generation unit 504 is configured to, in response to the successful verification of the execution results of each sub-task agent, control the video generation agent to generate the video based on the execution results of each sub-task agent.

[0078] In this embodiment, the specific processing of the allocation unit 501, control unit 502, verification unit 503, and generation unit 504 of the video generation device 500 based on multi-agent collaboration can be referred to Figure 2 The corresponding steps are 201, 202, 203 and 204 in the embodiment.

[0079] In some optional implementations of this embodiment, the device 500 further includes an adjustment unit (not shown in the figures), configured to: adjust the prompt word of the subtask agent that failed the execution result verification in response to the failure of any subtask agent; control the subtask agent that failed the verification to re-execute the corresponding subtask based on the adjusted prompt word, and update the execution result; perform consistency verification on the updated execution result; and control the video generation agent to generate a video based on the execution result that succeeded in the consistency verification in response to the updated execution result.

[0080] In some optional implementations of this embodiment, the allocation unit 501 is further configured to: determine the global framework of the video generation task by performing high-level global reasoning through a hierarchical thinking chain mechanism based on the demand information; and decompose the video generation task into multiple sub-tasks based on the global framework, and determine the description information of each sub-task.

[0081] In some optional implementations of this embodiment, multiple subtasks include: an outline script subtask; and the control unit 502 is further configured to: based on the description information of the outline script subtask, control the outline script agent to determine the outline of the video through a high-level thought chain; and based on the outline, control the outline script agent to determine the plot and dialogue through a low-level thought chain.

[0082] In some optional implementations of this embodiment, multiple subtasks include: character setting and scene subtasks; and the control unit 502 is further configured to: control the character setting and scene intelligent agent to determine character description information and scene description information through a hierarchical thinking chain mechanism based on the plot and the plot reasoning process information, wherein the scene description information includes the spatial coordinate system position of the character and / or the spatial coordinate system position of the object.

[0083] In some optional implementations of this embodiment, multiple subtasks include: a character reference image subtask; and the control unit 502 is further configured to: control the character reference image agent based on character description information to determine the character reference image.

[0084] In some optional implementations of this embodiment, the control unit 502 is further configured to: control the character reference image agent based on the character description information to determine multiple initial prompt words; determine the initial prompt word to be optimized from the multiple initial prompt words; control the character reference image agent to optimize the initial prompt word to be optimized; and control the character reference image agent to determine the character reference image based on the initial prompt word that does not need optimization and the optimized initial prompt word.

[0085] In some optional implementations of this embodiment, the control unit 502 is further configured to: control the character reference image agent to cross-fuse the initial prompt words that do not require optimization to determine the core feature words; control the character reference image agent to perform mutation optimization on the initial prompt words to be optimized to determine the mutated words; control the character reference image agent to determine the optimized initial prompt words based on the core feature words and mutated words through a genetic algorithm.

[0086] In some optional implementations of this embodiment, multiple subtasks include: storyboarding task; and the control unit 502 is further configured to: control the storyboarding agent to generate storyboards based on the outline, character description information, scene description information and character reference images.

[0087] In some optional implementations of this embodiment, multiple subtasks include a storyboard design subtask and a storyboard generation subtask; and the control unit 502 is further configured to: based on plot, dialogue, character description information, and scene description information, control the storyboard design agent to determine the total number of storyboards and the overall rhythm through a high-level thought chain; based on the total number of storyboards and the overall rhythm, control the storyboard design agent to determine the storyboard design scheme through a low-level thought chain; and based on the storyboard design scheme and character reference images, control the storyboard generation agent to generate storyboards.

[0088] In some optional implementations of this embodiment, the control unit 502 is further configured to: control the storyboard generation agent to generate prompts based on the storyboard design scheme and character reference images; control the storyboard generation agent to optimize the prompts based on a genetic algorithm to determine the optimized prompts; and control the storyboard generation agent to generate storyboards based on the optimized prompts and character reference images.

[0089] In some optional implementations of this embodiment, multiple subtasks include a storyboard quality inspection subtask; and the control unit 502 is further configured to: control the storyboard quality inspection agent to perform quality inspection on the storyboard; in response to detecting an unqualified storyboard, control the storyboard generation agent to re-optimize the prompt words based on a genetic algorithm to determine new prompt words; and based on the new prompt words and character reference images, control the storyboard generation agent to generate storyboards.

[0090] In some optional implementations of this embodiment, multiple subtasks include a storyboard quality inspection subtask; and the control unit 502 is further configured to: control the storyboard quality inspection agent to perform quality inspection on the storyboard diagrams; in response to detecting unqualified storyboard diagrams, control the storyboard design agent to redetermine the storyboard design scheme; and based on the redetermined storyboard design scheme and character reference images, control the storyboard diagram generation agent to generate storyboard diagrams.

[0091] In some optional implementations of this embodiment, the generation unit 504 is further configured to: control the video generation agent to generate dialogue based on the dialogue and character description information; control the video generation agent to generate video footage based on the storyboard; control the video generation agent to determine background sound effects based on the plot; and control the video generation agent to synthesize a video based on the dialogue, video footage, and background sound effects.

[0092] In some optional implementations of this embodiment, the verification unit 503 is further configured to: verify the consistency between the execution results of each subtask agent; and / or verify the consistency between the requirement information and the execution results of each subtask agent.

[0093] In some optional implementations of this embodiment, the apparatus further includes a determining unit (not shown in the figures), configured to: convert audio in the promotional video into text content; and determine demand information based on the text content.

[0094] In some optional implementations of this embodiment, the device further includes a splicing unit (not shown in the figures), configured to splice the generated video as a title sequence with the promotional video to generate the target video.

[0095] In some optional implementations of this embodiment, the control unit 502 is further configured to: for a target subtask, based on the description information of the target subtask, run multiple large language models in parallel to determine multiple execution results of the target subtask; and determine the best execution result from the multiple execution results of the target subtask as the execution result of the target subtask.

[0096] The collection, storage, use, processing, transmission, provision, and disclosure of any type of information, such as user personal information, in this technical solution comply with relevant laws and regulations and do not violate public order and good morals.

[0097] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0098] An electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method described in process 200.

[0099] A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method described in process 200.

[0100] A computer program product includes a computer program that, when executed by a processor, implements the method described in process 200.

[0101] Figure 6 A schematic block diagram of an example electronic device 600 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0102] like Figure 6 As shown, device 600 includes a computing unit 601, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 602 or a computer program loaded from storage unit 608 into random access memory (RAM) 603. RAM 603 may also store various programs and data required for the operation of device 600. The computing unit 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0103] Multiple components in device 600 are connected to I / O interface 605, including: input unit 606, such as keyboard, mouse, etc.; output unit 607, such as various types of monitors, speakers, etc.; storage unit 608, such as disk, optical disk, etc.; and communication unit 609, such as network card, modem, wireless transceiver, etc. Communication unit 609 allows device 600 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0104] The computing unit 601 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 601 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 performs the various methods and processes described above, such as a video generation method based on multi-agent cooperation. For example, in some embodiments, the video generation method based on multi-agent cooperation can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 608. In some embodiments, part or all of the computer program can be loaded and / or installed on device 600 via ROM 602 and / or communication unit 609. When the computer program is loaded into RAM 603 and executed by the computing unit 601, one or more steps of the video generation method based on multi-agent cooperation described above can be performed. Alternatively, in other embodiments, computing unit 601 may be configured by any other suitable means (e.g., by means of firmware) to perform a video generation method based on multi-agent cooperation.

[0105] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0106] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0107] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0108] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0109] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0110] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0111] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0112] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A video generation method based on multi-agent cooperation, comprising: Based on the requirements information, the description information of multiple sub-tasks for generating the video is determined. These sub-tasks include: generating an outline script, determining character settings and scenes, generating character reference images, and generating storyboard images. Based on the description information of each subtask, the corresponding subtask agents are controlled to determine the execution results of each subtask, including: based on the description information of the outline script generation subtask, the outline script agent is controlled to determine the outline of the video through a high-level thought chain; based on the outline, the outline script agent is controlled to synchronize its own plot reasoning process to subsequent agents through a low-level thought chain, providing a logical basis for the design of character settings, scenes, and storyboards, reducing irrelevant plot from the source, and determining the plot and lines. Among these, the outline script agent selects multiple models for generating text content, generates outline scripts in parallel, and selects the best plot and lines through evaluation by the overall control agent. Consistency verification is performed on the execution results of each subtask agent, including: the matching degree between the plot logic and product requirements, the adaptability between character design and scene, and the coherence between storyboard and outline. In response to the successful verification of the execution results of each subtask agent, the video generation agent is controlled to generate the video based on the execution results of each subtask agent.

2. The method according to claim 1, wherein, The method further includes: In response to the failure of the execution result verification of any subtask agent, adjust the prompt word of the subtask agent that failed the verification. Based on the adjusted prompt words, the subtask agent that failed the verification is controlled to re-execute the corresponding subtask and update the execution result; Perform a consistency check on the updated execution results; In response to the successful consistency check of the updated execution result, the video generation agent is controlled to generate the video based on the successful consistency check result.

3. The method according to claim 1, wherein, The description information for determining multiple sub-tasks used to generate the video based on the demand information includes: Based on the demand information, a high-level global reasoning mechanism is used to determine the global framework of the video generation task. Based on the global framework, the video generation task is broken down into multiple sub-tasks, and the description information of each sub-task is determined.

4. The method according to claim 1, wherein, The multiple sub-tasks include: the character setting scenario sub-task; and The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: Based on the plot and the reasoning process information of the plot, the control character and scene intelligent agent determines the character description information and scene description information through a hierarchical thinking chain mechanism. The scene description information includes the spatial coordinate system position of the character and / or the spatial coordinate system position of the object.

5. The method according to claim 4, wherein, The multiple sub-tasks include: the character reference image sub-task; and The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: Based on the character description information, the intelligent agent controlling the character reference image determines the character reference image.

6. The method according to claim 5, wherein, The step of controlling the character reference image agent based on the character description information to determine the character reference image includes: Based on the character description information, control the character reference image agent to determine multiple initial prompt words; Determine the initial prompt word to be optimized from the plurality of initial prompt words; The intelligent agent controlling the character reference image optimizes the initial prompt words to be optimized; Based on the initial prompts that do not require optimization and the optimized initial prompts, the intelligent agent controlling the character reference image determines the character reference image.

7. The method according to claim 6, wherein, The control agent in the reference graph optimizes the initial prompts to be optimized, including: The control agent based on the character reference image cross-fusions the initial prompts that do not require optimization to determine the core feature words; The intelligent agent controlling the character reference image mutates and optimizes the initial prompt words to be optimized, and determines the mutated words; The intelligent agent controlling the character reference image determines the optimized initial prompt words based on the core feature words and the variant words through a genetic algorithm.

8. The method according to claim 5, wherein, The multiple sub-tasks include: the mirroring task; and The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: Based on the outline, the character description information, the scene description information, and the character reference image, the storyboard agent is controlled to generate storyboard images.

9. The method according to claim 5, wherein, The multiple sub-tasks include a storyboard design sub-task and a storyboard generation sub-task; as well as The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: Based on the plot, the lines, the character descriptions, and the scene descriptions, the storyboard design agent determines the total number of storyboards and the overall rhythm through a high-level thought chain. Based on the total number of storyboards and the overall rhythm, the storyboard design agent is controlled to determine the storyboard design scheme through a low-level thought chain; Based on the storyboard design scheme and the character reference image, a storyboard-generating agent is controlled to generate the storyboard.

10. The method according to claim 9, wherein, The process of controlling the storyboard generation agent based on the storyboard design scheme and the character reference image to generate storyboard images includes: Based on the storyboard design scheme and the character reference image, control the storyboard to generate an intelligent agent and generate prompt words; The intelligent agent that generates the storyboard based on the genetic algorithm optimizes the prompt words and determines the optimized prompt words; Based on the optimized prompts and the character reference image, the storyboard generation agent is controlled to generate the storyboard.

11. The method according to claim 10, wherein, The multiple sub-tasks include the storyboard quality inspection sub-task; as well as The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: Control the storyboard quality inspection intelligent agent to inspect the storyboard; In response to the detection of an unqualified storyboard, the storyboard generation agent is controlled by a genetic algorithm to re-optimize the prompt words and determine new prompt words; Based on the new prompt words and the character reference image, a storyboard generation agent is controlled to generate storyboards.

12. The method according to claim 9, wherein, The multiple sub-tasks include the storyboard quality inspection sub-task; as well as The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: Control the storyboard quality inspection intelligent agent to inspect the storyboard; In response to the detection of an unqualified storyboard, the storyboard design agent is controlled to redetermine the storyboard design scheme; Based on the redefined storyboard design and the character reference image, a storyboard-generating agent is controlled to generate the storyboard.

13. The method according to claim 8, wherein, The process of controlling the video generation agent based on the execution results of each sub-task agent to generate video includes: Based on the dialogue and the character description information, control the video generation intelligent agent to generate the dialogue voice; Based on the storyboard, control the video generation agent to generate video frames; Based on the aforementioned plot, control the video to generate an intelligent agent and determine the background sound effects; Based on the spoken dialogue, the video footage, and the background sound effects, a video-generating intelligent agent is controlled to synthesize the video.

14. The method according to claim 1, wherein, The consistency verification of the execution results of each sub-task agent includes: Verify the consistency between the execution results of each subtask agent; and / or The consistency between the required information and the execution results of each subtask agent is verified.

15. The method according to claim 1, wherein, The method further includes: Convert the audio in promotional videos into text content; The required information is determined based on the text content.

16. The method according to claim 15, wherein, The method further includes: The generated video is used as the intro and spliced ​​with the promotional video to generate the target video.

17. The method according to any one of claims 1-16, wherein, The process of controlling the corresponding subtask agents based on the description information of each subtask and determining the execution result of each subtask includes: For the target subtask, based on the description information of the target subtask, multiple large language models are run in parallel to determine multiple execution results of the target subtask; The best execution result is determined from the multiple execution results of the target subtask and taken as the execution result of the target subtask.

18. A video generation device based on multi-agent cooperation, comprising: The allocation unit is configured to determine the description information of multiple sub-tasks for generating the video based on the demand information. The multiple sub-tasks include: generating an outline script, determining character settings and scenes, generating character reference images, and generating storyboard images. The control unit is configured to control the corresponding subtask agents based on the description information of each subtask, and to determine the execution results of each subtask. This includes: based on the description information of the outline script generation subtask, controlling the outline script agent to determine the outline of the video through a high-level thought chain; based on the outline, controlling the outline script agent to synchronize its own plot reasoning process to subsequent agents through a low-level thought chain, providing a logical basis for the design of characters, scenes, and storyboards, reducing irrelevant plot from the source, and determining the plot and dialogue. The outline script agent selects multiple models for generating text content, generates outline scripts in parallel, and selects the best plot and dialogue through evaluation by the overall control agent. The verification unit is configured to perform consistency verification on the execution results of each subtask agent, including: the matching degree between the plot logic and product requirements, the adaptability between character design and scene, and the coherence between storyboard and outline. The generation unit is configured to respond to the successful verification of the execution results of each subtask agent, and control the video generation agent to generate the video based on the execution results of each subtask agent.

19. A video generation system based on multi-agent cooperation, comprising: A central control agent is configured to perform the method according to any one of claims 1-17; The subtask agent is configured to determine the execution result of the subtask based on the description information of the subtask determined by the overall control agent. The video generation agent is configured to generate videos based on the execution results of the subtask agent.

20. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-17.

21. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-17.

22. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-17.

Images