A Heterogeneous AI Agent Multimedia Content Collaborative Generation Method Based on Task Decomposition

Abstract

This invention relates to the technical fields of artificial intelligence and software engineering, specifically to a method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition. The method includes: standardizing requirement information to obtain a directed acyclic task graph; analyzing the matching degree between the execution requirements of each node in the task graph and the execution capabilities of each agent in the agent set to obtain agent scheduling information; controlling agents based on the agent scheduling information and the task graph to obtain multiple task execution results, wherein multiple agents share a target workspace, which includes a structural layer, a text layer, and a vector layer; and fusing the multiple task execution results to obtain the target multimedia content. This invention, by analyzing the matching degree between agents and graph nodes, can effectively leverage the advantages of different agents and utilize the target workspace for collaborative constraints among different agents, thereby improving the content consistency and compliance of AIGC solutions.

Description

Technical Field

[0001] This invention relates to the technical field of artificial intelligence and software engineering, specifically to a method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition. Background Technology

[0002] Existing AI-generated content (AIGC) systems mostly use a single model for direct processing or a fixed pipeline.

[0003] Among them, the single-model approach is difficult to ensure "consistent settings, coherent structure, unified style, and compliant control" in long, multi-role, and multi-channel tasks at the same time; while the fixed pipeline approach can process in stages, it lacks a dynamic division of labor mechanism based on task attributes and model capabilities, and the sharing and consistency control of intermediate results are insufficient, which can easily lead to problems such as "setting drift, conflicting standards, sudden style changes, and inconsistencies".

[0004] It is evident that the content generation effect of the AIGC solution provided by existing technologies is poor. Summary of the Invention

[0005] The purpose of this invention is to provide a method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition, in order to solve the technical problem of poor content generation effect of AIGC solutions provided by the prior art.

[0006] In a first aspect, one embodiment of the present invention provides a method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition, the method comprising:

[0007] The requirement information is standardized to obtain a task graph, wherein the task graph is a directed acyclic graph;

[0008] The matching degree between the execution requirements of each graph node in the task graph and the execution capabilities of each agent in the agent set is analyzed to obtain agent scheduling information, wherein the agent scheduling information is used to indicate the agent in the agent set whose execution capability matches the execution requirements of the corresponding graph node to the highest degree.

[0009] Based on the agent scheduling information and the task graph, agent control is performed to obtain multiple task execution results. The agent set includes multiple agents sharing a target workspace, which includes a structure layer, a text layer, and a vector layer.

[0010] The results of the multiple tasks are merged to obtain the target multimedia content.

[0011] In one embodiment, the step of standardizing the requirement information to obtain a task graph includes:

[0012] The requirement information is correlated with the corresponding directed acyclic architecture graph to obtain the original graph, wherein the original graph is a directed acyclic graph and the number of graph nodes in the original graph is less than or equal to the number of graph nodes in the directed acyclic architecture graph.

[0013] The computational complexity of each graph node in the original graph is analyzed based on the demand information to obtain node evaluation information.

[0014] Based on the node evaluation information, the graph nodes included in the original graph are adjusted to obtain the task graph, wherein the computational complexity of any graph node in the task graph is less than a first complexity threshold and greater than a second complexity threshold.

[0015] In one embodiment, the computational complexity of a graph node is determined based on the text prediction length of the result of the task it corresponds to, the number of key objects included in the task description it corresponds to, the number of output branches it corresponds to, the number of output constraints it corresponds to, and the number of output modal types it corresponds to.

[0016] In one embodiment, the node adjustment includes: node splitting, node merging, and globally aligned node insertion.

[0017] In one embodiment, the computing resources corresponding to each graph node in the task graph are positively correlated with its computational complexity.

[0018] In one embodiment, the matching score between the target graph node and the target agent is determined based on the corresponding vector similarity index, the unit cost of calling the target agent, the corresponding task processing latency, the corresponding task processing effect index, and the corresponding agent switching cost. Here, the target graph node is any graph node included in the task graph, and the target agent is any agent included in the agent set. The matching score is used to indicate the degree of matching between the execution requirements of the corresponding graph node and the execution capabilities of the corresponding agent. The vector similarity index is used to represent the degree of similarity between the requirement vector of the corresponding graph node and the capability vector of the corresponding agent. The task processing effect index is used to represent the reliability of the corresponding agent in processing the task indicated by the corresponding graph node.

[0019] In one embodiment, the step of performing agent control based on the agent scheduling information and the task graph to obtain multiple task execution results includes:

[0020] The graph node corresponding to the target agent is determined based on the agent scheduling information, wherein the target agent is any agent in the set of agents;

[0021] Based on the target workspace, the target agent is controlled to execute the tasks indicated by the corresponding graph nodes to obtain initial execution results;

[0022] In the event of a conflict between the initial execution result and the constraints recorded in the target workspace, the dependent agent and the target agent are coordinated to execute the task indicated by the corresponding graph node based on the target workspace control, so as to obtain the task execution result of the graph node corresponding to the target agent, wherein the graph node corresponding to the dependent agent is the predecessor node of the graph node corresponding to the target agent;

[0023] If the initial execution result does not conflict with the constraints recorded in the target workspace, the initial execution result is determined as the task execution result of the graph node corresponding to the target agent.

[0024] In one embodiment, the step of cooperating with the target agent to execute the task indicated by the corresponding graph node based on the target workspace control dependent agent to obtain the task execution result of the graph node corresponding to the target agent includes:

[0025] Based on the conflict between the initial execution result and the constraints recorded in the target workspace, and the target workspace, the control-dependent agent re-executes the task indicated by its corresponding graph node to obtain input adjustment information;

[0026] Based on the input adjustment information and the target workspace control, the target agent re-executes the task indicated by its corresponding graph node to obtain the task execution result of the graph node corresponding to the target agent.

[0027] In one embodiment, the results of the multiple task executions are fused to obtain target multimedia content, including:

[0028] The results of the multiple task executions are merged, structural rearranged, and content merged to obtain initial merged content.

[0029] The initial fusion content is optimized based on the user profile corresponding to the demand information to obtain optimized fusion content;

[0030] Based on the output modal conditions corresponding to the demand information, the optimized fusion content is format-converted to obtain the target multimedia content.

[0031] In one embodiment, the structure layer is used to record the constraints corresponding to the requirement information and the structural information derived when the agent processes the corresponding task; the text layer is used to record the reference text used to assist the agent in processing the task and the content text formed when the agent processes the corresponding task; and the vector layer is used to record the index vectors of the data in the structure layer and the text layer.

[0032] Secondly, another embodiment of the present invention provides a heterogeneous AI agent multimedia content collaborative generation system based on task decomposition, the system comprising:

[0033] The standard processing module is used to standardize the requirement information to obtain a task graph, wherein the task graph is a directed acyclic graph;

[0034] The scheduling module is used to analyze the degree of matching between the execution requirements of each graph node in the task graph and the execution capabilities of each agent in the agent set, so as to obtain agent scheduling information. The agent scheduling information is used to indicate the agent in the agent set whose execution capability matches the execution requirements of the corresponding graph node the most.

[0035] The task execution module is used to perform agent control based on the agent scheduling information and the task graph to obtain multiple task execution results. The agent set includes multiple agents sharing a target workspace, which includes a structure layer, a text layer and a vector layer.

[0036] The content generation module is used to merge the results of the multiple task executions to obtain the target multimedia content.

[0037] Thirdly, in another embodiment of the present invention, an electronic device is provided, including a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps of the method described in the first aspect.

[0038] Fourthly, in another embodiment of the present invention, a computer-readable storage medium is provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0039] The present invention has the following beneficial effects:

[0040] This invention transforms demand information into a directed acyclic graph (DAG). Through the task nodes and dependencies between them, it achieves modular and visual processing of demand information. Then, based on the matching degree between the agent's execution capability and the task node's execution requirements, it assigns the most suitable agent to each task node for processing. This fully leverages the content generation advantages of different agents in the agent set. During the execution of corresponding tasks, a target workspace is configured as a knowledge-sharing space for different agents. By utilizing the structural, text, and vector layers included in the target workspace, intermediate results are shared and content constraints are applied to the agent's task execution operations. This ensures that the content style of the different task execution results corresponding to each task node remains consistent and the content logic is continuous, thereby improving the content generation effect of the target multimedia content in response to demand information. Attached Figure Description

[0041] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating a method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition, provided in an embodiment of the present invention.

[0043] Figure 2 This is a schematic diagram illustrating a Task DAG example and its parallel execution relationship provided in an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of a closed-loop process provided in an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of the structure of a heterogeneous AI intelligent agent multimedia content collaborative generation system based on task decomposition provided in an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0047] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a task decomposition-based heterogeneous AI agent multimedia content collaborative generation method proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0048] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0049] The following description, in conjunction with the accompanying drawings, details the specific scheme of a task-decomposition-based heterogeneous AI agent multimedia content collaborative generation method provided by the present invention.

[0050] In one embodiment, the present invention provides a method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition, such as... Figure 1 As shown, the method includes:

[0051] Step S1: Standardize the requirement information to obtain the task diagram.

[0052] The task graph is a directed acyclic graph.

[0053] In this invention, the demand information is used to represent the content generation needs in multimedia content generation application scenarios. For example, in brand and marketing content generation scenarios, the demand information can indicate that content adapted to different dissemination channels (such as short video scripts, WeChat official account articles, and advertising copy) can be automatically generated for a specific brand, ensuring that the brand tone, core selling points, and prohibited expressions remain consistent across different content. In long story and script creation scenarios, the demand information can indicate that story creation should be carried out based on a specific outline or designated theme, and that character settings, worldview rules, and timelines should be uniformly managed to avoid setting conflicts during multiple generation processes. In government publicity and emergency science popularization content generation scenarios, the demand information can indicate that designated policy documents or emergency plans should be transformed into public science popularization content with a clear structure, consistent wording, and adaptability to multiple channels, avoiding information ambiguity and logical conflicts.

[0054] The above-mentioned standardization process includes at least the content decomposition of the demand information (to form graph nodes) and content logic analysis (to form edges connecting different graph nodes).

[0055] Specifically, the steps for standardizing the requirements information to obtain the task graph include:

[0056] The requirement information is correlated with the corresponding directed acyclic architecture graph to obtain the original graph, wherein the original graph is a directed acyclic graph and the number of graph nodes in the original graph is less than or equal to the number of graph nodes in the directed acyclic architecture graph.

[0057] The computational complexity of each graph node in the original graph is analyzed based on the demand information to obtain node evaluation information.

[0058] Based on the node evaluation information, the graph nodes included in the original graph are adjusted to obtain the task graph, wherein the computational complexity of any graph node in the task graph is less than a first complexity threshold and greater than a second complexity threshold.

[0059] In the above settings, the computational complexity of each graph node is compared with the value of a preset threshold (i.e., the first complexity threshold and the second complexity threshold) to achieve dynamic adjustment of the nodes. This avoids the situation where overly complex graph nodes (graph nodes with computational complexity greater than or equal to the first complexity threshold) are assigned to the agent, and avoids the waste of computational resources and content interference caused by overly simple graph nodes (graph nodes with computational complexity less than or equal to the second complexity threshold).

[0060] In this invention, multiple directed acyclic architecture diagrams are pre-configured, and these multiple directed acyclic architecture diagrams correspond to multiple different content generation scenarios. After determining the content generation scenario corresponding to the requirement information, the directed acyclic architecture diagram corresponding to the requirement information can be determined accordingly.

[0061] The directed acyclic architecture diagram specifically indicates the full range of processing tasks included in the corresponding content generation scenario and the dependencies between each processing task. For example, the directed acyclic architecture diagram corresponding to the brand and marketing content generation scenario includes processing tasks such as: selling point extraction, creative main slogan, short video script / storyboard, WeChat official account long article generation, poster compression, compliance review, and unified style and terminology solidification.

[0062] The above directed acyclic architecture diagram can be obtained based on expert experience, or it can be obtained by merging multiple task flows in the corresponding scenario generated from historical records.

[0063] It should be noted that the above-mentioned association analysis operation specifically involves identifying graph nodes that are associated with the requirement information in the directed acyclic architecture graph corresponding to the requirement information.

[0064] The association determination between requirement information and the corresponding graph nodes of the directed acyclic architecture can be accomplished through semantic vector extraction and semantic vector similarity analysis. Graph nodes whose semantic vector similarity (e.g., measured using cosine similarity) between requirement information is greater than a preset similarity threshold are identified as graph nodes associated with requirement information. It should be understood that in this case, graph nodes whose semantic vector similarity is less than or equal to the similarity threshold will not be retained in the original graph.

[0065] The computational complexity mentioned above is used to illustrate the difficulty of completing the task indicated by the corresponding graph node. The higher the computational complexity, the higher the difficulty of completing the task indicated by the corresponding graph node.

[0066] The node adjustments include: node splitting, node merging, and global alignment node insertion.

[0067] Specifically, node splitting indicates the process of breaking down a single complex graph node into several simpler graph nodes; while node merging indicates the process of integrating multiple related or consecutive simple graph nodes into a relatively complex graph node; as for global alignment node insertion (such as indicating the insertion of nodes for tasks like compliance review, unified style and glossary solidification), it indicates the process of inserting predefined graph nodes for coordination, synchronization or quality control at specific locations in the graph structure. The former two aim to balance the computational effect and resource overhead during subsequent agent processing, while the latter is used to control the coordination, data alignment or quality compliance of the entire system.

[0068] For example, the node decomposition process can be a process of breaking down complex scene nodes into simpler dialogue sub-nodes, narration sub-nodes, and stage instruction sub-nodes; or, a process of breaking down complex channel rewriting nodes into simpler short video narration nodes, subtitle nodes, and cover title nodes; or, a process of breaking down complex worldview setting nodes into simpler era background setting nodes, geographical environment setting nodes, power structure setting nodes, and rule setting nodes.

[0069] In applications, the node adjustment strategy can be supplemented by semantic island risk judgment logic. When a graph node with a computational complexity less than or equal to the second complexity threshold is identified, it can be further judged whether the graph node has semantic island risk. Only when the graph node is judged to have semantic island risk will node merging or global alignment node insertion be performed. When the graph node is judged not to have semantic island risk, the computational complexity of the graph node is forcibly updated to a set value between the first complexity threshold and the second complexity threshold, and the hyperparameters associated with the computational complexity of the graph node are corrected according to the graph node and its metadata.

[0070] Specifically, if the similarity between the semantic vector corresponding to a graph node and the semantic vector of the global topic corresponding to the demand information is less than a set similarity threshold, it can be determined that the graph node has a semantic island risk; otherwise, it can be determined that the graph node does not have a semantic island risk.

[0071] Furthermore, the computational complexity of a graph node is determined based on the text prediction length of the corresponding task execution result, the number of key objects included in the corresponding task description, the number of corresponding output branches, the number of corresponding output constraints, and the number of corresponding output modal types.

[0072] Based on the above settings, the computational complexity of each graph node can be fully and accurately quantified, thereby ensuring the effective execution of node adjustment work.

[0073] For example, in the task graph, the first The computational complexity of each graph node It can be represented as:

[0074]

[0075] in, The text prediction length of the task execution result for the corresponding graph node (which can be predicted by a neural network); Indicates the number of key objects (such as story characters or abstract entities) included in the task description of the corresponding graph node; This refers to the number of output branches corresponding to the corresponding graph node (such as the number of interactive branches, the number of endings, the number of channel versions, etc.). Indicates the number of cross-scene references or the number of foreshadowing events to be resolved for the corresponding graph node; Indicates the number of output constraints corresponding to the corresponding graph node (such as brand tone, prohibited expressions, glossary, consistency of terminology, etc.). This indicates the number of output modalities (such as text / image / audio / video script, etc.) corresponding to a given graph node; , , , , , These are all configurable weight hyperparameters used to adjust the contribution of each indicator to computational complexity.

[0076] Furthermore, the computational resources corresponding to each node in the task graph are positively correlated with its computational complexity.

[0077] The aforementioned computing resources include, but are not limited to: the window length used for contextual retrieval and the number of search entries returned when retrieving related information.

[0078] In other words, the higher the computational complexity of a graph node, the larger its window length for context retrieval, and the more retrieval entries returned when retrieving related information. This is to adapt to situations where complex tasks require more auxiliary information for reference and learning, while adaptively reducing the overhead of simple tasks in information retrieval. This ensures the task execution effect of each graph node while minimizing the overall task execution cost.

[0079] Step S2: Analyze the degree of matching between the execution requirements of each graph node in the task graph and the execution capabilities of each agent in the agent set, in order to obtain agent scheduling information.

[0080] Among them, agent scheduling information is used to indicate the agent in the agent set whose execution capability matches the execution requirements of the corresponding graph node to the highest degree.

[0081] Specifically, the matching score between the target graph node and the target agent is determined based on the corresponding vector similarity index, the unit cost of calling the target agent, the corresponding task processing latency, the corresponding task processing effect index, and the corresponding agent switching cost. Here, the target graph node is any graph node included in the task graph, and the target agent is any agent included in the agent set. The matching score is used to indicate the degree of matching between the execution requirements of the corresponding graph node and the execution capabilities of the corresponding agent. The vector similarity index is used to represent the degree of similarity between the requirement vector of the corresponding graph node and the capability vector of the corresponding agent. The task processing effect index is used to represent the reliability of the corresponding agent in processing the task indicated by the corresponding graph node.

[0082] In this invention, the demand vector can be understood as a vector representation of the execution requirements of the corresponding graph node, and the capability vector can be understood as a vector representation of the execution capabilities of the corresponding agent.

[0083] In one example, the aforementioned requirement vectors include, but are not limited to: entity density, dialogue weight, target style / register, channel type, length, external retrieval requirements, and deadline latency; the aforementioned capability vectors include, but are not limited to: quality score on the standard subtask benchmark, context window, tool usage capability, inference cost, inference latency, and inference stability.

[0084] For example, in the task graph, the first The set of graph nodes and agents of the th... Matching score between agents It can be represented as:

[0085]

[0086] in, Instruction No. The demand vector for each graph node. Instruction No. A capability vector of an agent Indicates the inner product or vector similarity between the two. Instruction to call the Unit cost of an intelligent agent; Indicates the prediction of the first The agent processes the first The task processing latency of the task indicated by each graph node; Indicates the first The task processing of the intelligent agent The task processing performance index of each graph node (which can be understood as the historical success rate of the corresponding agent in processing the corresponding type of task, and can be obtained by weighting indicators such as consistency pass rate, rollback rate, and user acceptance rate). The instruction is to switch from the currently working agent to the [number]th agent. The cost of agent switching caused by an agent (usually the loss of contextual information). , , , These are the weighting coefficients.

[0087] When batch assignment (i.e., the process of obtaining agent scheduling information), heuristics (such as greedy selection by score) or linear assignment (such as maximum weight matching) can be used to allocate multiple tasks (i.e. graph nodes) to multiple agents in order to maximize the overall benefits and meet resource upper limit constraints (such as concurrency, budget, and latency limits).

[0088] Preferably, the task processing performance index is learned online using Bayesian updates or multi-armed gambling strategies (such as Thompson Sampling) to gradually select a better combination of agents in different scenarios.

[0089] To avoid policy oscillations, a hysteresis mechanism is introduced: when a candidate agent's score relative to the currently applied agent ( ) Improve the consistency and controllability of multi-round generation by selecting not to switch agents when the score is below a set threshold; or, set a minimum dwell time and force that the agent switching logic will only be determined after the continuous use time of each agent exceeds the minimum dwell time.

[0090] The calculation formula for the agent switching cost can be expressed as:

[0091]

[0092] In the above formula, This is used to represent the cost and time overhead of the pre-cue token required to re-inject the context (such as character settings and plot summary) related to the current task into agent m. This is used to represent the probability that the content output style will undergo an unexpected change after switching to agent m based on historical data. By introducing agent switching costs, high-frequency agent switching actions performed during task scheduling to pursue small quality gains in a single task can be effectively prevented, thereby ensuring the consistency and stability of the overall style and narrative in long, multi-step generation tasks.

[0093] Step S3: Perform agent control based on the agent scheduling information and the task graph to obtain multiple task execution results.

[0094] The intelligent agent set includes multiple intelligent agents sharing a target workspace (also known as a three-layer blackboard), which includes a structure layer, a text layer, and a vector layer.

[0095] Specifically, the step of performing agent control based on the agent scheduling information and the task graph to obtain multiple task execution results includes:

[0096] The graph node corresponding to the target agent is determined based on the agent scheduling information, wherein the target agent is any agent in the set of agents;

[0097] Based on the target workspace, the target agent is controlled to execute the tasks indicated by the corresponding graph nodes to obtain initial execution results;

[0098] In the event of a conflict between the initial execution result and the constraints recorded in the target workspace, the dependent agent and the target agent are coordinated to execute the task indicated by the corresponding graph node based on the target workspace control, so as to obtain the task execution result of the graph node corresponding to the target agent, wherein the graph node corresponding to the dependent agent is the predecessor node of the graph node corresponding to the target agent;

[0099] If the initial execution result does not conflict with the constraints recorded in the target workspace, the initial execution result is determined as the task execution result of the graph node corresponding to the target agent.

[0100] Further, the step of cooperating with the target agent to execute the task indicated by the corresponding graph node based on the target workspace control dependent agent to obtain the task execution result of the graph node corresponding to the target agent includes:

[0101] Based on the conflict between the initial execution result and the constraints recorded in the target workspace, and the target workspace, the control-dependent agent re-executes the task indicated by its corresponding graph node to obtain input adjustment information;

[0102] Based on the input adjustment information and the target workspace control, the target agent re-executes the task indicated by its corresponding graph node to obtain the task execution result of the graph node corresponding to the target agent.

[0103] Based on the above settings, by introducing constraints, the output of task execution results that disrupt global consistency is limited (such as task execution results that violate scenario constraints, or task execution results that logically conflict with the already output task execution results). At the same time, with the assistance of dependent intelligent agents, the global consistency of the regenerated task execution results is ensured by adaptively correcting the task input information, so as to ensure that the content style of different task execution results corresponding to each task node remains consistent, the content logic is continuous, and the content is compliant.

[0104] It should be noted that in the application, in order to ensure the global consistency of the task execution results of the graph nodes corresponding to the target agent, the target agent may repeatedly execute the task indicated by its corresponding graph node. In this process, the rewriting operation is completed by using the least-cost backtracking rewriting method to minimize the resource overhead of task re-execution. That is, the dependent agent needs to be determined iteratively (that is, starting from the corresponding graph node, the previous graph nodes are traced back step by step from the nearest to the farthest and information is regenerated in coordination. The dependent agent determined in each iteration is different). The specific iteration logic is as follows: only when the dependent agent determined in the current iteration is unable to compensate for the loss of global consistency of the candidate execution results output by the target agent in executing the task, the next iteration is carried out. And based on the graph node corresponding to the dependent agent used in the current iteration, forward tracing is performed in the task graph (only one node needs to be traced back), and the agent corresponding to the traced graph node and the dependent agent used in the current iteration are jointly determined as the dependent agent used in the next iteration.

[0105] In this invention, the loss in global consistency of the candidate execution result can be determined by comprehensively considering the computational resource overhead of predicting and rewriting to generate the candidate execution result, the computational time overhead of predicting and rewriting to generate the candidate execution result, and the risk of the predicted candidate execution result damaging the already evaluated task execution result. The risk of the predicted candidate execution result damaging the already evaluated task execution result can be determined adaptively based on the number of dependencies of the candidate execution result on other already evaluated task execution results. That is, the more dependencies there are, the greater the corresponding risk of damage.

[0106] For example, when conflicts such as role attribute conflicts, event timeline conflicts, or brand terminology conflicts are identified in the candidate execution results, it is determined that the dependent agent determined in the corresponding iteration cannot compensate for the loss of global consistency in the candidate execution results output by the target agent.

[0107] It should be noted that the constraints recorded in the target workspace may include: static constraints preloaded before task processing (which can be obtained by analyzing requirement information), and dynamic constraints generated based on the task execution results that have been generated and passed consistency audit.

[0108] For example, in the scenario of popularizing emergency response plans, static constraints can be understood as pre-determined safe haven locations. If the suggested haven location output by a candidate execution result is inconsistent with the safe haven location, then the candidate execution result can be considered to not meet the constraints. Alternatively, if the generated and consistent task execution result indicates that a certain role is male (dynamic constraint), while a candidate execution result indicates that the role is female, then the candidate execution result can also be considered to not meet the constraints.

[0109] In other words, the target workspace not only supports pre-loaded static constraints (such as background constraints, sensitive word constraints, etc.), but also supports dynamic constraints formed after the corresponding graph nodes are processed (such as role attribute constraints, event time constraints, etc.).

[0110] In this invention, the structure layer is used to record the constraints corresponding to the requirement information and the structural information derived when the agent processes the corresponding task; the text layer is used to record the reference text used to assist the agent in processing the task and the content text formed when the agent processes the corresponding task; and the vector layer is used to record the index vectors of the data in the structure layer and the text layer.

[0111] The structure layer can be understood as a strong constraint (also known as a hard constraint) on the candidate execution results, including but not limited to: worldview, roles, events, and relationships, encompassing timelines, attribute domains, value ranges, and constraint rules. The text layer, on the other hand, can be understood as a weak constraint on the candidate execution results, including but not limited to: scene fragments, style tags, and review comments, supporting versioning and diffing (for version rollback and retrospective auditing). The vector layer is used to support the agent's access to information from the structure and text layers. In application, the structure layer, combined with a rule engine, enables content review of candidate execution results, thereby identifying type / relationship / timeline rule conflicts. The text layer, combined with NLI contradiction detection, embedding similarity thresholds, referential resolution / entity link consistency checks, and other measures, enables semantic conflict and drift judgment of candidate execution results, ensuring the logical consistency of the final output of multiple task execution results.

[0112] Hard constraint validation based on the structural layer is accomplished through a configurable rule engine. This engine supports defining constraints using domain-specific languages ​​(DSLs) or declarative rules. For example, it can specify that a role's profession must be one of several defined professions (such as doctor, lawyer, and teacher) or specify prohibited sensitive words (such as national level, money, etc.) in the content. The engine can perform real-time pattern matching and logical validation on new entries written to the blackboard (i.e., the target workspace). The rules set here are decoupled from the business logic to facilitate the maintenance and updating of the rule knowledge base by operations personnel.

[0113] The soft-constraint semantic detection based on the text layer is as follows: First, newly appearing paragraphs to be detected in the text are identified and linked to entity nodes defined in the blackboard structure layer. Then, for statements involving the same entity or event that may have logical conflicts (e.g., "A is in the hospital" and "A is at the airport"), the NLI model is called to determine their relationship (contradiction / neutrality / implication). Next, the paragraph to be detected is vectorized with the specified "style anchor" paragraph in the blackboard or the historical version description of the entity, and cosine similarity is calculated to assess whether semantic or style drift has occurred. Finally, a semantic consistency risk score is output, combining the risks of logical conflict and style deviation, and the paragraph to be detected is judged based on this score to determine whether it passes semantic detection (this can be done through threshold comparison). It should be noted that when new entries are added to the text layer or vector layer involving entities, relationships, or events, the corresponding nodes and edges in the structure layer can be automatically extracted and updated. If a conflict occurs, consistency detection is triggered and the corresponding update operation is prevented. When multiple versions of the same entity or event exist in the text layer, they can be prioritized based on version timestamps, generated agent weights, and user feedback ratings. Manual annotation and merging are also supported. In addition, whenever the structure layer or text layer is updated, the newly added or modified entries will be encoded into vectors in real time and entered into the vector layer for subsequent use.

[0114] In addition, multimodal processing nodes can be added to adapt to multimodal scenarios (such as image storyboard generation and audio script adaptation), and the style and semantic constraints of multimodal elements can be uniformly managed in the target workspace to achieve cross-modal consistency control (such as associating "visual bright style" with "vectorization of text cheerful tone").

[0115] Step S4: Merge the results of the multiple task executions to obtain the target multimedia content.

[0116] Specifically, step S4 includes:

[0117] The results of the multiple task executions are merged, structural rearranged, and content merged to obtain initial merged content.

[0118] The initial fusion content is optimized based on the user profile corresponding to the demand information to obtain optimized fusion content;

[0119] Based on the output modal conditions corresponding to the demand information, the optimized fusion content is format-converted to obtain the target multimedia content.

[0120] The above-mentioned structural rearrangement process can be understood as: analyzing the internal logic of the results of multiple tasks (such as time sequence, importance, causal relationship), breaking the isolated structure of the results of individual tasks, and planning a new, unified content skeleton. For example, inserting data charts into the corresponding arguments of the text outline and reserving space for video materials.

[0121] The content integration process can be understood as: connecting, transitioning, deduplicating, and supplementing the content of multiple task execution results to ensure content coherence.

[0122] The content optimization process based on user profiles can be understood as follows: adjusting wording based on the user's knowledge level reflected in the user profile (such as identifying and explaining professional terms that may exist in the initial fused content, or refining overly colloquial content in the initial fused content, etc.), and adjusting the level of detail in the content based on the user's preferences reflected in the user profile (strengthening and refining the logic of the content that users care about, while appropriately deleting the content that users do not care about).

[0123] The format conversion process can be understood as: the process of optimizing and integrating content based on the output modality conditions to convert information of the corresponding modality, such as converting text content into PPT format content through AI tools, or converting text content into video format content through AI tools.

[0124] Based on the above settings, the task execution results (such as text, images, audio, etc.) generated collaboratively by multiple intelligent agents are deeply integrated through structural rearrangement and content fusion to eliminate redundancy and contradictions between different contents, ensuring the semantic coherence and style consistency of the initial fused content. On this basis, targeted optimization based on user profiles is introduced to adjust language style, emotional tendency, example reference and cultural context based on user profiles to improve the dissemination effect of optimized fused content. Finally, the corresponding format conversion is performed based on the output modality conditions to adapt to the content format constraints of the corresponding channels and ensure that the format matching standard of the target multimedia content is set.

[0125] For ease of understanding, the following example is provided:

[0126] In one example, such as Figure 2 and Figure 3As shown, the heterogeneous AI agent multimedia content collaborative generation method based on task decomposition described in this invention specifically includes: the user setting the required content demand information (i.e., corresponding...) Figure 2 (Set nodes in the middle), and process the demand information in a structured way (i.e., corresponding to) Figure 2 The task graph is formed by constructing nodes in the task graph. The execution requirements (represented by requirement vectors) of each node in the task graph are then analyzed to determine the matching degree between the execution capabilities (represented by energy) of each agent in the agent set. This determines the agent corresponding to each node in the task graph, and the agent is controlled to complete the task indicated by the corresponding node, resulting in multiple task execution results (i.e., corresponding...). Figure 2 The results of multiple tasks (A, B, and C) are then merged (i.e., the corresponding results are merged). Figure 2 (merging nodes in the process), and perform consistency checks in conjunction with the target workspace (i.e., corresponding nodes). Figure 2 The output is then sent out after the detection node in the middle.

[0127] It utilizes three blackboard layers (structure layer, content layer, and vector layer) as the aforementioned target workspace for different intelligent agents to share information, manage versions, and support traceability.

[0128] In the above process, the execution result of the agent is written into three layers of blackboards to achieve adaptive updates of the structure layer, content layer, and vector layer. The execution result is checked for consistency by the static and dynamic constraints stored in the three layers of blackboards to identify possible rule conflicts or semantic conflicts. If the check passes, it is determined to be a deliverable output result. It is then merged with other execution results that have passed the check and the format is adjusted to become the content output result of the corresponding channel. If the check fails (i.e., a conflict is found), the rewrite subset is located and backtracked for rewriting (refer to the rewriting process based on the dependent agent mentioned above; to avoid repetition, it will not be repeated here). When there are multiple options for rewriting, the rewriting scheme with the lowest cost is selected first. If the execution result is not as expected, the rewriting scheme with the second lowest cost is selected. This process is iterated until a rewritten execution result that can pass the check is obtained. When the rewriting scheme is iteratively called, the number of graph nodes included (inserted) in the rewriting scheme increases continuously.

[0129] In summary, this invention proposes an engineering modeling method based on a task dependency graph (Task DAG) with standard nodes and executable contracts (i.e., standardizing requirement information to form a task graph). This transforms the subjective and ambiguous content creation process into a computable, schedulable, auditable, and adjustable task network composed of explicit input / output contracts, failure fallback strategies, and typified dependency edges. Compared with the fixed pipeline schemes used in related technologies, this method supports more flexible content generation measures. For example, when "style drift" is detected, "style enhancement and alignment nodes" can be inserted online, and when "timeline conflict" is detected, "timeline inference and correction nodes" can be inserted and the dependencies of relevant scene nodes can be adjusted. This design more realistically simulates the process of human content creation, thus achieving better content generation results.

[0130] Furthermore, this invention also constructs a quantification mechanism for the matching degree between demand vectors and capability vectors (i.e., analyzing the matching degree between the execution requirements of each graph node in the task graph and the execution capabilities of each agent in the agent set to obtain agent scheduling information), realizing deep complementary collaboration between heterogeneous agents. This mechanism not only achieves static adaptation between tasks and agents through vector matching, but also comprehensively considers quality benefits, costs, latency, historical success rates, and context switching costs. It can effectively suppress output style jumps and context breaks caused by frequently changing agents in pursuit of single-point optimization (constrained by switching penalty terms), strongly ensuring the logical coherence of long process generation. At the same time, it uses online learning (such as Thompson Sampling) to dynamically update the historical success rate confidence interval of agents, enabling the system to balance "exploring the potential of new agents" and "utilizing existing high-success-rate agents" to maximize long-term benefits.

[0131] In addition, a "three-layer blackboard" structure (i.e., the setting of the target workspace) was designed, and a joint consistency control mechanism of hard constraint rules and soft constraint semantics was integrated. Specifically, the structure layer (knowledge graph) manages strong constraints, the text layer (versioned fragments) records the process, and the vector layer (embedded index) supports semantic retrieval. This provides a unified and versioned source of facts for multi-agent collaboration. The innovative application of RAG (Retrieval Enhanced Generation) technology to the "memory backtracking" of the system is also applied. That is, before generation or rewriting, relevant historical settings, style anchors and high-quality fragments are automatically retrieved from the blackboard vector layer to form a prompt context, thereby giving the system the ability to continuously call and follow the existing settings in "long-term memory", effectively preventing settings from being forgotten or drifting.

[0132] It also implements a "minimum-cost backtracking and rewriting intelligent conflict resolution strategy." When a consistency check detects a conflict, instead of a simple retrieval or global rollback, it performs reverse impact analysis based on the task dependency graph and version graph to accurately locate the smallest subset of nodes causing the conflict. The "minimum cost" is a multi-dimensional evaluation metric that considers computational cost, additional latency, and the risk of secondary damage to already approved content. By selecting the backtracking path with the lowest cost for rewriting, it can repair consistency conflicts while maximizing the protection of valid content that has passed the evaluation, significantly reducing the overall cost and content fluctuation of collaborative generation.

[0133] Meanwhile, this invention also establishes a self-correction mechanism for node decomposition granularity, solving the problems of "over-decomposition" and "semantic silos" in task decomposition. Through a recursive splitting / merging strategy with dual threshold control, and combined with semantic silo risk quantification criteria (such as global topic deviation, missing dependency references, and weak cross-node consistency), the task granularity is dynamically adjusted.

[0134] In one embodiment, the present invention provides a heterogeneous AI agent multimedia content collaborative generation system based on task decomposition, such as... Figure 4 As shown, the system 200 includes:

[0135] The standard processing module 201 is used to standardize the requirement information to obtain a task graph, wherein the task graph is a directed acyclic graph;

[0136] The scheduling module 202 is used to analyze the degree of matching between the execution requirements of each graph node in the task graph and the execution capabilities of each agent in the agent set, so as to obtain agent scheduling information. The agent scheduling information is used to indicate the agent in the agent set whose execution capability matches the execution requirements of the corresponding graph node the most.

[0137] The task execution module 203 is used to perform agent control based on the agent scheduling information and the task graph to obtain multiple task execution results. The agent set includes multiple agents sharing a target workspace, which includes a structure layer, a text layer and a vector layer.

[0138] The content generation module 204 is used to fuse the results of the multiple task executions to obtain the target multimedia content.

[0139] It should be noted that the system provided in the above embodiments is only an example of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device can be divided into different functional modules to complete all or part of the functions described above. In addition, the heterogeneous AI intelligent agent multimedia content collaborative generation system based on task decomposition and the heterogeneous AI intelligent agent multimedia content collaborative generation method based on task decomposition provided in the above embodiments belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0140] This invention also provides an electronic device. Please refer to [link to relevant documentation]. Figure 5 The electronic device may include a processor 301, a memory 302, and a program 3021 stored in the memory 302 and capable of running on the processor 301.

[0141] When program 3021 is executed by processor 301, it can achieve the following: Figure 1 Any steps in the corresponding method embodiments and the achievement of the same beneficial effects will not be repeated here.

[0142] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by hardware related to program instructions, and the program can be stored in a readable medium.

[0143] This invention also provides a readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described functions. Figure 1 Any step in the corresponding method embodiment can achieve the same technical effect, and will not be repeated here to avoid repetition.

[0144] The computer-readable storage medium of this invention can be any combination of one or more computer-readable media. The computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. For example, a computer-readable storage medium can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0145] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of sending, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0146] The program code contained on the storage medium can be transmitted using any suitable medium, including but not limited to wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0147] Computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or terminal. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0148] This invention also provides a computer program product that, when run on a computer, causes the computer to perform the aforementioned steps to achieve the task decomposition-based heterogeneous AI agent multimedia content collaborative generation method provided in the above embodiments.

[0149] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0150] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition, characterized in that, The method includes: The requirement information is standardized to obtain a task graph, wherein the task graph is a directed acyclic graph; The matching degree between the execution requirements of each graph node in the task graph and the execution capabilities of each agent in the agent set is analyzed to obtain agent scheduling information, wherein the agent scheduling information is used to indicate the agent in the agent set whose execution capability matches the execution requirements of the corresponding graph node to the highest degree. Based on the agent scheduling information and the task graph, agent control is performed to obtain multiple task execution results. The agent set includes multiple agents sharing a target workspace. The target workspace includes a structure layer, a text layer, and a vector layer. The structure layer is used to record the constraints corresponding to the requirement information and the structural information derived when the agent processes the corresponding task. The text layer is used to record the reference text used to assist the agent in processing the task and the content text formed when the agent processes the corresponding task. The vector layer is used to record the index vectors of the data in the structure layer and the text layer. The results of the multiple tasks are merged to obtain the target multimedia content.

2. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 1, characterized in that, The steps to standardize the requirements information to obtain the task diagram include: The requirement information is correlated with the corresponding directed acyclic architecture graph to obtain the original graph, wherein the original graph is a directed acyclic graph and the number of graph nodes in the original graph is less than or equal to the number of graph nodes in the directed acyclic architecture graph. The computational complexity of each graph node in the original graph is analyzed based on the demand information to obtain node evaluation information. Based on the node evaluation information, the graph nodes included in the original graph are adjusted to obtain the task graph, wherein the computational complexity of any graph node in the task graph is less than a first complexity threshold and greater than a second complexity threshold.

3. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 2, characterized in that, The computational complexity of a graph node is determined based on the text prediction length of the corresponding task execution result, the number of key objects included in the corresponding task description, the number of corresponding output branches, the number of corresponding output constraints, and the number of corresponding output modal types.

4. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 2, characterized in that, The node adjustments include: node splitting, node merging, and global alignment node insertion.

5. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 2, characterized in that, The computational resources corresponding to each node in the task graph are positively correlated with its computational complexity.

6. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 1, characterized in that, The matching score between the target graph node and the target agent is determined based on the corresponding vector similarity index, the unit cost of calling the target agent, the corresponding task processing latency, the corresponding task processing effect index, and the corresponding agent switching cost. Here, the target graph node is any graph node included in the task graph, and the target agent is any agent included in the agent set. The matching score is used to indicate the degree of matching between the execution requirements of the corresponding graph node and the execution capabilities of the corresponding agent. The vector similarity index is used to represent the degree of similarity between the requirement vector of the corresponding graph node and the capability vector of the corresponding agent. The task processing effect index is used to represent the reliability of the corresponding agent in processing the task indicated by the corresponding graph node.

7. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 1, characterized in that, The steps of performing agent control based on the agent scheduling information and the task graph to obtain multiple task execution results include: The graph node corresponding to the target agent is determined based on the agent scheduling information, wherein the target agent is any agent in the set of agents; Based on the target workspace, the target agent is controlled to execute the tasks indicated by the corresponding graph nodes to obtain initial execution results; In the event of a conflict between the initial execution result and the constraints recorded in the target workspace, the dependent agent and the target agent are coordinated to execute the task indicated by the corresponding graph node based on the target workspace control, so as to obtain the task execution result of the graph node corresponding to the target agent, wherein the graph node corresponding to the dependent agent is the predecessor node of the graph node corresponding to the target agent; If the initial execution result does not conflict with the constraints recorded in the target workspace, the initial execution result is determined as the task execution result of the graph node corresponding to the target agent.

8. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 7, characterized in that, The steps for obtaining the task execution result of the graph node corresponding to the target agent based on the target workspace control dependent agent and the target agent collaboratively executing the task indicated by the corresponding graph node include: Based on the conflict between the initial execution result and the constraints recorded in the target workspace, and the target workspace, the control-dependent agent re-executes the task indicated by its corresponding graph node to obtain input adjustment information; Based on the input adjustment information and the target workspace control, the target agent re-executes the task indicated by its corresponding graph node to obtain the task execution result of the graph node corresponding to the target agent.

9. The method for collaborative generation of multimedia content by heterogeneous AI agents based on task decomposition according to claim 1, characterized in that, The results of the multiple task executions are fused to obtain the target multimedia content, including: The results of the multiple task executions are merged, structural rearranged, and content merged to obtain initial merged content. The initial fusion content is optimized based on the user profile corresponding to the demand information to obtain optimized fusion content; Based on the output modal conditions corresponding to the demand information, the optimized fusion content is format-converted to obtain the target multimedia content.

Images