Visual arrangement system oriented to multiple modes and multiple agents
By combining a visual orchestration module and an adaptive intelligent orchestration engine with graph neural networks and reinforcement learning, the problem of rigid orchestration and low collaborative efficiency in multimodal and multi-agent tasks of existing platforms is solved, achieving efficient and dynamic task execution and optimization, and enhancing the interpretability and user control of the system.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN ZHONGXING XINYUN SERVICE CO LTD
- Filing Date
- 2025-12-26
- Publication Date
- 2026-04-28
AI Technical Summary
Existing AI development platforms suffer from problems such as rigid orchestration methods, weak multimodal processing capabilities, primitive multi-agent collaboration mechanisms, passive human-machine collaboration modes, and insufficient interpretability and controllability when handling complex multimodal and multi-agent collaborative tasks.
It employs a visual orchestration module, a multimodal data bus module, and an adaptive intelligent orchestration engine. Task processes are defined through a drag-and-drop graphical interface. Dynamic and adaptive task orchestration and execution are achieved using graph neural networks and reinforcement learning decision networks. It supports multimodal data processing and agent collaboration, and provides a visual display of decision-making basis.
It enables efficient and dynamic execution and optimization of complex multimodal and multi-agent tasks, enhances the system's adaptability and interpretability, and improves user trust and control.
Smart Images
Figure CN121934831A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of artificial intelligence technology, and in particular to a visual orchestration system for multimodal and multi-agent systems. Background Technology
[0002] With the rapid development of artificial intelligence technology, especially the breakthrough in Large Language Model (LLM) capabilities, AI Agents have become a core technological path for automating complex tasks. Currently, AI development environments on the market are mainly divided into three categories: 1) Traditional workflow engines: such as Activiti, Camunda, etc. These platforms are often used for business process management (BPM), but they were not designed for AI. They have very limited capabilities in integrating AI models and processing multimodal data, and their process rules are rigid, lacking intelligence and flexibility. 2) General AI development frameworks: such as TensorFlow and PyTorch, which provide powerful underlying support for model training, but do not focus on upper-level application orchestration and multi-agent collaboration. They have high development thresholds and require a lot of coding work. 3) Emerging Agent Development Platforms: Such as Dify, Flowise, and Coze. These platforms lower the barrier to agent development through visualization and low-code methods, supporting the connection of LLM, tools, knowledge bases, and other elements into simple workflows. They are currently the mainstream technologies, but significant technical bottlenecks still exist when dealing with truly complex, dynamic, multimodal, and multi-agent collaborative tasks. Specifically: The orchestration method is rigid and lacks intelligence: Most existing platforms adopt the "what you see is what you get" execution mode, where the flowchart dragged out by the user is directly interpreted as a fixed execution path. This static orchestration cannot be dynamically adjusted according to real-time changes in task requirements, data characteristics or system load, resulting in poor adaptability and low efficiency in complex and ever-changing environments. Weak multimodal processing capabilities and severe information loss: Although some platforms support access to various types of data such as images and documents, they often adopt a simple "textualization" strategy (such as image description and document OCR), forcibly converting non-textual modalities into text before processing. This approach not only causes severe loss of original information (such as image details and audio intonation), but also fails to achieve true cross-modal deep fusion and collaborative understanding. The multi-agent collaboration mechanism is primitive and lacks global optimization: When dealing with complex tasks that require the cooperation of multiple agents, existing platforms usually adopt simple sequential calls or predefined branch logic, which is essentially a series of "individual intelligences". This leads to the "barrel effect", where the efficiency of the entire system is limited by the slowest link, and it cannot solve deep-seated collaboration problems such as resource competition and goal conflict between agents. The human-in-the-Loop (HIL) model is passive: In existing platforms, HIL is usually treated as a fixed step in the process, with humans passively reviewing or processing specific tasks. The system lacks the ability to autonomously determine when human intervention is needed, resulting in low collaboration efficiency and the inability to achieve intelligent triggering at key nodes. Insufficient explainability and controllability: The decision-making process of an agent is often a "black box," making it difficult for users to understand its behavioral logic and decision-making basis. When problems occur, debugging and optimization become extremely difficult, and users' trust and control over the system are greatly reduced. Summary of the Invention
[0003] The purpose of this invention is to provide a visual orchestration system for multimodal and multi-agent tasks. It deeply integrates user-friendly visual programming with a powerful backend intelligent optimization engine. Through an innovative three-layer architecture of "visualization-graphization-optimization", it achieves dynamic, adaptive orchestration and efficient execution of complex multimodal and multi-agent tasks.
[0004] To achieve the above objectives, the following technical solution is adopted: A visual orchestration system for multimodal and multi-agent systems, including The visual orchestration module includes a drag-and-drop graphical user interface that allows users to define a visual blueprint of the target task by selecting and connecting agent nodes, data source nodes, tool nodes, and control flow nodes. The Visual Process Compilation Module is used to receive the Visual Process Blueprint and parse and compile it into a Dynamic Task Diagram. The multimodal data bus module is used to uniformly encapsulate and transmit data flowing between nodes in a dynamic task graph; An adaptive intelligent orchestration engine is used to encode the current task graph state based on a graph neural network model during the execution of a dynamic task graph, so as to obtain a high-dimensional state vector representing the task graph. The state vector is then input into a reinforcement learning decision network to execute corresponding scheduling actions at multiple decision moments. The visualization module is used to drive the actual task execution on the dynamic task graph based on the scheduling actions output by the adaptive intelligent orchestration engine, and monitor the running status and performance indicators during the execution process. At the same time, it also presents the actual execution path, intelligent agent nodes and the specific models or resources they call on the visualization process blueprint interface in the form of highlighted paths or animations, and displays the decision basis information corresponding to the node or branch when the user selects a target node or branch.
[0005] Furthermore, the dynamic task graph includes This represents multiple task nodes and their topology that are invoked by each intelligent agent's task, data processing node, or tool. Multiple edges represent the data dependencies between task nodes, where each edge carries the data modality type and dependency information; The agent capability profile information associated with each task node includes the input or output modalities supported by the corresponding agent, historical performance indicators, and a list of optional models or tools. Variable structure information representing at least one optional execution branch and / or parallel execution path.
[0006] Furthermore, the specific operating steps of the multimodal data bus module are as follows: Encapsulate raw data of text, images, audio, and other modalities into multimodal data objects; Generate and store a corresponding vectorized representation and metadata information including data source, timestamp, and modality type for each multimodal data object; Data routing and cross-modal alignment between agents are achieved based on multimodal data objects.
[0007] Furthermore, the adaptive intelligent orchestration engine includes The graph state representation submodule is used to construct graph structure input features before each task scheduling based on the execution state of each task node in the current dynamic task graph, the feature vectors of each multimodal data object, and the load information of each computing resource node. The graph structure input features are then iteratively aggregated and updated through a graph neural network to obtain a high-dimensional graph vector representing the global task state. The reinforcement learning strategy submodule is used to map high-dimensional graph vectors to corresponding action probability distributions and value evaluation results, and to sample the actual actions to be executed based on the action probability distributions. The graph neural network supports incremental updates during task execution, so that local state updates are performed only on the affected subgraphs after some task nodes have been executed.
[0008] Furthermore, the scheduling actions in the adaptive intelligent orchestration engine include Select the next task node to be executed from multiple candidate task nodes or branch paths; Select a specific execution model or tool from among multiple candidate models or tools associated with a particular task node; Allocate computing resources to the target task nodes based on the current resource load of each computing node; When the decision confidence level is lower than a preset threshold, the human-machine collaboration node is triggered to interrupt automatic execution and request human intervention. The adaptive intelligent orchestration engine calculates a comprehensive reward based on the time taken, resource consumption and result quality after the task is completed, and updates the parameters of the reinforcement learning decision network online based on the comprehensive reward to continuously optimize the orchestration strategy for subsequent tasks.
[0009] Furthermore, the adaptive intelligent orchestration engine is further configured to, when triggering the human-machine collaboration node: The local structure of the current dynamic task graph, related multimodal data objects, and key information of candidate branch paths are abstracted and compressed to generate decision context information for human users. The decision context information is displayed to human users through a visual orchestration module, allowing them to manually select target branch paths or adjust task parameters. The selection results of human users are used as one of the external feedback signals for reinforcement learning strategies to participate in the calculation of comprehensive rewards and policy updates.
[0010] By adopting the above solution, the beneficial effects of the present invention are: This invention effectively solves the core pain points of existing technologies in handling complex multimodal and multi-agent tasks, and provides a powerful, efficient and easy-to-use development and operation platform system for building next-generation enterprise-level intelligent applications. Attached Figure Description
[0011] Figure 1 This is a system architecture diagram of the present invention. Detailed Implementation
[0012] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.
[0013] Reference Figure 1 As shown, the present invention provides a visual orchestration system for multimodal and multi-agent systems. In one embodiment, it includes... The visual orchestration module includes a drag-and-drop graphical user interface that allows users to define a visual blueprint of the target task by selecting and connecting agent nodes, data source nodes, tool nodes, and control flow nodes. The Visual Process Compilation Module is used to receive the Visual Process Blueprint and parse and compile it into a Dynamic Task Diagram. The multimodal data bus module is used to uniformly encapsulate and transmit data flowing between nodes in a dynamic task graph; An adaptive intelligent orchestration engine is used to encode the current task graph state based on a graph neural network model during the execution of a dynamic task graph, so as to obtain a high-dimensional state vector representing the task graph. The state vector is then input into a reinforcement learning decision network to execute corresponding scheduling actions at multiple decision moments. The visualization module is used to drive the actual task execution on the dynamic task graph based on the scheduling actions output by the adaptive intelligent orchestration engine, and monitor the running status and performance indicators during the execution process. At the same time, it also presents the actual execution path, intelligent agent nodes and the specific models or resources they call on the visualization process blueprint interface in the form of highlighted paths or animations, and displays the decision basis information corresponding to the node or branch when the user selects a target node or branch.
[0014] Users can define business process blueprints through a drag-and-drop visual interface. The system's built-in "visual process compilation module" automatically parses and compiles these blueprints into a dynamic task graph containing rich metadata and optimization space. Subsequently, an adaptive intelligent orchestration engine based on graph neural networks (GNN) and reinforcement learning (RL) takes over the graph. This engine can perceive the system status in real time (such as agent load, data characteristics, and resource status) and make dynamic and adaptive optimization decisions on agent selection, model invocation, resource allocation, and even the fusion method of multimodal data based on preset optimization goals (such as lowest cost, shortest latency, and highest accuracy). Ultimately, the system not only executes tasks efficiently but also visualizes the optimized actual execution path and decision basis, forming a complete and transparent human-machine collaborative development and optimization closed loop. Specifically: Since most existing technologies directly interpret and execute user-defined visual processes, resulting in rigid processes that cannot be dynamically adjusted, this invention adds a visual process compilation module to achieve automatic conversion from high-level business logic to a low-level optimizable computational graph. Users can use the system-provided node library, which includes various agents (such as data processing, model invocation, and logical judgment), data sources, tools, and control flows (such as branching and parallelism), to intuitively build the application's workflow blueprint through drag-and-drop and connection. After receiving the visual blueprint, the visual process compilation module does not generate a static execution script but parses and compiles it into a structured dynamic task graph. The dynamic task graph includes... This represents multiple task nodes and their topology that are invoked by each intelligent agent's task, data processing node, or tool. Multiple edges represent the data dependencies between task nodes, where each edge carries the data modality type and dependency information; The agent capability profile information associated with each task node includes the input or output modalities supported by the corresponding agent, historical performance indicators, and a list of optional models or tools. Variable structure information representing at least one optional execution branch and / or parallel execution path.
[0015] As mentioned above, the dynamic task graph not only contains nodes (Agents) and edges (data flows), but also includes richer meta-information, such as: Agent capability profile: Each agent node includes its functional description, supported input / output modalities, performance metrics (historical average latency, cost), available model options (e.g., GPT-4 vs. GPT-3.5), etc. Data dependencies: The edges are labeled with the modal type (text, image, vector, etc.) and dependencies of the data flow; Variable structure: The graph contains selectable parallel branches and alternative paths, providing a basis for subsequent dynamic decision-making.
[0016] This module can decouple the user's business intent from the underlying execution optimization, transforming the rigid "flowchart" into a "possibility network" full of optimization space, which is a prerequisite for realizing intelligent orchestration.
[0017] Preferably, the adaptive intelligent orchestration engine is based on a dynamic task graph and optimizes execution strategies in real time. The adaptive intelligent orchestration engine includes... The graph state representation submodule is used to construct graph structure input features before each task scheduling based on the execution state of each task node in the current dynamic task graph, the feature vectors of each multimodal data object, and the load information of each computing resource node. The graph structure input features are then iteratively aggregated and updated through a graph neural network to obtain a high-dimensional graph vector representing the global task state. The reinforcement learning strategy submodule is used to map high-dimensional graph vectors to corresponding action probability distributions and value evaluation results, and to sample the actual actions to be executed based on the action probability distributions. The graph neural network supports incremental updates during task execution, so that local state updates are performed only on the affected subgraphs after some task nodes have been executed.
[0018] The scheduling actions in the adaptive intelligent orchestration engine include Select the next task node to be executed from multiple candidate task nodes or branch paths; Select a specific execution model or tool from among multiple candidate models or tools associated with a particular task node; Allocate computing resources to the target task nodes based on the current resource load of each computing node; When the decision confidence level is lower than a preset threshold, the human-machine collaboration node is triggered to interrupt automatic execution and request human intervention. The adaptive intelligent orchestration engine calculates a comprehensive reward based on the time taken, resource consumption and result quality after the task is completed, and updates the parameters of the reinforcement learning decision network online based on the comprehensive reward to continuously optimize the orchestration strategy for subsequent tasks.
[0019] When the adaptive intelligent orchestration engine is triggered at the human-machine collaboration node, it is further configured as follows: The local structure of the current dynamic task graph, related multimodal data objects, and key information of candidate branch paths are abstracted and compressed to generate decision context information for human users. The decision context information is displayed to human users through a visual orchestration module, allowing them to manually select target branch paths or adjust task parameters. The selection results of human users are used as one of the external feedback signals for reinforcement learning strategies to participate in the calculation of comprehensive rewards and policy updates.
[0020] In summary, addressing the core pain points of existing system orchestration logic being rigid and lacking intelligence, this invention constructs an adaptive intelligent orchestration engine. At each step of task execution, the engine first encodes the current dynamic task graph state using a Graph Neural Network (GNN). The GNN effectively captures the complex topology and dependencies between nodes in the graph, compressing the entire task graph (including the current state of each agent, data flow characteristics, resource load, etc.) into a high-dimensional vector as a comprehensive and accurate mathematical representation of the current system state. Subsequently, this state vector is input into a pre-trained reinforcement learning (RL) decision network. The goal of the RL agent is to learn an optimal policy to maximize cumulative rewards (e.g., "completing the task efficiently and at low cost"). At each decision point, the action space of the RL agent includes: Agent selection: When there are multiple parallel or optional paths, select the next agent to execute; Model / Tool Selection: Select a specific execution backend for an Agent. For example, choose to call the expensive GPT-4 for steps that require high accuracy, and choose an inexpensive local model for simple summary tasks. Resource scheduling: Assign computationally intensive tasks (such as image generation) to nodes with GPUs, and assign I / O intensive tasks to nodes with good network conditions; Human-machine collaboration trigger: When the RL Agent determines that the confidence level of the current decision is lower than a certain threshold based on the state representation output by the GNN, the process can be automatically paused and the "Human-Machine Collaboration (HITL)" node can be triggered to request human intervention and decision-making.
[0021] After the task is completed, the system calculates a comprehensive reward value based on the actual cost, latency, and result quality of the execution. This reward signal is then backpropagated through algorithms such as policy gradient to update the parameters of the RL network. This means that the orchestration engine can continuously learn from past execution experience and constantly optimize its scheduling strategy, becoming "smarter" with use. For example, it can learn to prioritize low-latency paths during peak business periods and low-cost paths during off-peak nighttime periods.
[0022] Preferably, in order to solve the problems of information loss and alignment difficulties when multimodal data flows between agents, the present invention constructs a multimodal data bus module, the working steps of which are as follows: Encapsulate raw data of text, images, audio, and other modalities into multimodal data objects; Generate and store a corresponding vectorized representation and metadata information including data source, timestamp, and modality type for each multimodal data object; Data routing and cross-modal alignment between agents are achieved based on multimodal data objects.
[0023] All data within the system, regardless of its original modality (text, image, audio, etc.), is encapsulated in a standardized data container (MultimodalDataObject). This object not only contains the original data but also mandates its vectorized representation (Embedding) and metadata (such as source, timestamp, and modality type). When the output of one agent needs to be used as input for another agent of a different modality, the data bus and orchestration engine work together. For example, an image analysis agent outputs a MultimodalDataObject containing the description "a golden retriever on the grass." The downstream text summarization agent can directly utilize the text description within it. More importantly, the intelligent orchestration engine can use GNN to analyze the vector similarity of multiple MultimodalDataObjects in the image, identifying whether they point to the same real-world entity. This allows the engine to decide on cross-modal fusion operations, such as combining image information with relevant user comment text and feeding it into a large multimodal model for more accurate intent understanding.
[0024] Preferably, the present invention also includes a visualization module. The present invention emphasizes the two-way nature of visualization, which is used not only for "inputting" the task flow, but also for "outputting" and explaining the execution process. Specifically, after the task is executed, the system will clearly display the execution path actually selected by the intelligent orchestration engine, the specific model called, and the resources on the original visualization blueprint in the form of highlights or animation. When the user clicks on a decision node (such as a selected branch), the system will display the decision basis in a simple and easy-to-understand way, for example: "Due to the current high system load and the task priority of 'medium', the 'low latency' path has been automatically selected, and the 'high precision' path has been abandoned." This interpretability greatly enhances the user's trust in and control over the system.
[0025] In one specific embodiment, an intelligent analysis system for the financial investment research field is built based on the system of the present invention. This system can automatically process multimodal financial information (announcements, research reports, news, financial charts) and generate structured investment research reports. The specific system construction and process are as follows: 1) Agent Definition and Registration: Define and register the following core agents in the system: Multimodal data acquisition agent: Responsible for scraping information such as company announcements (PDF), news (text), and financial reports (including charts) from the internet and databases; Financial Statement Chart Analysis Agent: Built-in image recognition model, specifically designed to extract key financial data tables from financial statement images; Core Idea Extraction Agent: Utilizes LLM analysis to extract key ideas and data from text content; Risk Event Detection Agent: Analyzes news and announcements to identify potential negative events; Investment Research Report Generation Agent: Integrates all information and generates structured investment research reports according to preset templates.
[0026] 2) Application process orchestration: On the visual interface, the above agents can be connected by dragging and dropping to build a parallel information processing process. The data collection agent is the starting point, and its output is sent to three agents at the same time: financial statement chart analysis, core viewpoint extraction and risk event detection. After these three agents process in parallel, the structured data is output to the investment research report generation agent to finally form the report. 3) Intelligent Orchestration and Execution: When a new analysis task is started, the intelligent orchestration engine starts working. It dynamically selects the data source based on the real-time API call cost and latency. For example, when processing financial reports, if the chart parsing agent is found to have low confidence due to image blurring, the engine will automatically trigger Human-Machine Collaboration (HITL) to request the analyst to manually enter or confirm the data. At the same time, the engine will allocate computationally intensive model inference tasks to idle GPU servers based on server load. 4) Results Presentation and Optimization: After the task is completed, the system not only generates an investment research report, but also highlights the actual execution path on the visualization interface and marks the nodes and reasons that trigger human intervention. The analyst's modifications and feedback to the report will be used as new training data to optimize the RL strategy online, enabling the system to make better decisions in future tasks.
[0027] In another specific embodiment, a system capable of automatically generating product marketing content (copy, posters, short video scripts) is built based on the system of the present invention. The system construction and process are as follows: 1) Agent Definition and Registration: Define and register the following core agents: Product Information Understanding Agent: Input a product link, and it will parse its basic information, user reviews (text), and product images. (Image); Selling Point Extraction Agent: Based on product information and user reviews, summarize the core selling points; Marketing copy generation agent: Generates marketing copy in different styles based on selling points; Poster Design Agent: Utilizes text-to-image models (such as Stable Diffusion) to combine product images and marketing copy. Generate marketing posters; Short video script generation agent: Converts text and images into storyboard scripts.
[0028] 2) Application Process Orchestration: Construct a process of "product understanding -> selling point extraction -> multi-channel content generation". The output of the selling point extraction agent will be distributed to three parallel generation agents: copywriting, poster, and video script. 3) Intelligent orchestration and execution: During execution, the orchestration engine will face the decision of model selection. For example, for core products, the engine will choose to call the high-quality but more expensive text-based image model to generate posters; for long-tail products, it may choose a faster and lower-cost model; if the product image quality is not good, the engine will first call an image enhancement agent (as a dynamic insertion step) to process it, and then proceed with the subsequent poster generation. This dynamic task flow reorganization is something that traditional static workflows cannot achieve.
[0029] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A visual orchestration system for multimodal, multi-agent systems, characterized in that, include The visual orchestration module includes a drag-and-drop graphical user interface that allows users to define a visual blueprint of the target task by selecting and connecting agent nodes, data source nodes, tool nodes, and control flow nodes. The Visual Process Compilation Module is used to receive the Visual Process Blueprint and parse and compile it into a Dynamic Task Diagram. The multimodal data bus module is used to uniformly encapsulate and transmit data flowing between nodes in a dynamic task graph; An adaptive intelligent orchestration engine is used to encode the current task graph state based on a graph neural network model during the execution of a dynamic task graph, so as to obtain a high-dimensional state vector representing the task graph. The state vector is then input into a reinforcement learning decision network to execute corresponding scheduling actions at multiple decision moments. The visualization module is used to drive the actual task execution on the dynamic task graph based on the scheduling actions output by the adaptive intelligent orchestration engine, and monitor the running status and performance indicators during the execution process. At the same time, it also presents the actual execution path, intelligent agent nodes and the specific models or resources they call on the visualization process blueprint interface in the form of highlighted paths or animations, and displays the decision basis information corresponding to the node or branch when the user selects a target node or branch.
2. The visualization orchestration system for multimodal and multi-agent systems according to claim 1, characterized in that, The dynamic task graph includes This represents multiple task nodes and their topology that are invoked by each intelligent agent's task, data processing node, or tool. Multiple edges represent the data dependencies between task nodes, where each edge carries the data modality type and dependency information; The agent capability profile information associated with each task node includes the input or output modalities supported by the corresponding agent, historical performance indicators, and a list of optional models or tools. Variable structure information representing at least one optional execution branch and / or parallel execution path.
3. The visualization orchestration system for multimodal and multi-agent systems according to claim 1, characterized in that, The specific working steps of the multimodal data bus module are as follows: Encapsulate raw data of text, images, audio, and other modalities into multimodal data objects; Generate and store a corresponding vectorized representation and metadata information including data source, timestamp, and modality type for each multimodal data object; Data routing and cross-modal alignment between agents are achieved based on multimodal data objects.
4. The visualization orchestration system for multimodal and multi-agent systems according to claim 1, characterized in that, The adaptive intelligent orchestration engine includes The graph state representation submodule is used to construct graph structure input features before each task scheduling based on the execution state of each task node in the current dynamic task graph, the feature vectors of each multimodal data object, and the load information of each computing resource node. The graph structure input features are then iteratively aggregated and updated through a graph neural network to obtain a high-dimensional graph vector representing the global task state. The reinforcement learning strategy submodule is used to map high-dimensional graph vectors to corresponding action probability distributions and value evaluation results, and to sample the actual actions to be executed based on the action probability distributions. The graph neural network supports incremental updates during task execution, so that local state updates are performed only on the affected subgraphs after some task nodes have been executed.
5. The visualization orchestration system for multimodal and multi-agent systems according to claim 4, characterized in that, The scheduling actions in the adaptive intelligent orchestration engine include Select the next task node to be executed from multiple candidate task nodes or branch paths; Select a specific execution model or tool from among multiple candidate models or tools associated with a particular task node; Allocate computing resources to the target task nodes based on the current resource load of each computing node; When the decision confidence level is lower than a preset threshold, the human-machine collaboration node is triggered to interrupt automatic execution and request human intervention. The adaptive intelligent orchestration engine calculates a comprehensive reward based on the time taken, resource consumption and result quality after the task is completed, and updates the parameters of the reinforcement learning decision network online based on the comprehensive reward to continuously optimize the orchestration strategy for subsequent tasks.
6. The visualization orchestration system for multimodal multi-agent systems according to claim 5, characterized in that, When the adaptive intelligent orchestration engine is triggered at the human-machine collaboration node, it is further configured as follows: The local structure of the current dynamic task graph, related multimodal data objects, and key information of candidate branch paths are abstracted and compressed to generate decision context information for human users. The decision context information is displayed to human users through a visual orchestration module, allowing them to manually select target branch paths or adjust task parameters. The selection results of human users are used as one of the external feedback signals for reinforcement learning strategies, participating in the calculation of comprehensive rewards and policy updates.