Agent development system and method based on visual arrangement and large model cooperation

Abstract

This invention discloses an intelligent agent development system and method based on visual orchestration and large-scale model collaboration, belonging to the field of artificial intelligence technology. It includes several steps: visual orchestration, graph state perception, reinforcement learning dynamic decision-making, experience playback and reflection analysis, and online policy update. First, a workflow is built and a topology description file is exported. The topology information is parsed and a global dependency representation is generated. Candidate paths are evaluated based on the representation, and the optimal path is selected. Abnormal trajectories are backtracked and analyzed to generate an optimization strategy. The optimization strategy is verified in an isolated environment, and once confirmed to be effective, it is fixed to the workflow configuration. This invention uses a graph attention network to deeply encode the workflow topology, enabling the system to perceive the cascading coupling relationships between nodes. Through reinforcement learning, multi-dimensional scheduling and collaborative optimization are achieved, ensuring that the system dynamically adjusts its execution strategy according to real-time task characteristics, significantly improving the orchestration adaptability and operational efficiency in complex long-link scenarios.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an intelligent agent development system and method based on visual orchestration and large model collaboration. Background Technology

[0002] In recent years, large language model technology has made groundbreaking progress. Pre-trained models based on the Transformer architecture have demonstrated outstanding capabilities in natural language understanding and generation tasks. Against this backdrop, how to utilize large models to build intelligent agents capable of perceiving the environment, planning and making decisions, and executing tasks has become a research hotspot in the application layer of artificial intelligence. Currently, developers mainly rely on two paradigms to build intelligent agent workflows: one is the hard-coded development mode, which directly calls the application programming interface provided by the large model vendor, concatenates prompt words in a chain in the business code, and processes the model's returned results; the other is to use open-source frameworks such as LangChain to define multi-step calling logic by writing scripts. Both methods use a directed acyclic graph as the underlying execution model, encapsulating the large model call, knowledge retrieval, external tool execution, and condition judgment into independent nodes. By defining the dependencies between nodes and the data flow, a complete workflow is built. This graph-based orchestration paradigm has significantly improved upon single model calls and can handle complex scenarios containing branching and convergence logic. However, as intelligent agent applications expand deeper into enterprise-level long-chain scenarios, existing solutions still have room for improvement in terms of topology awareness, scheduling flexibility, and system self-optimization level.

[0003] Existing workflow orchestration schemes based on directed acyclic graphs (DAGs) have limited ability to perceive the topological coupling relationships between nodes during execution. Traditional schemes typically complete the static topology analysis of the workflow during the compilation phase, determining the execution order and parallel boundaries based on dependencies. However, during runtime, there is a lack of quantitative assessment of the criticality of each node in the topology and the cascading effects of upstream and downstream nodes. When the workflow contains a large number of nodes, the response latency or output quality fluctuation of a certain intermediate node may spread step by step along the data dependency path, ultimately having a significant impact on the overall execution effect. Due to the lack of quantitative analysis of the propagation of node influence and graph-level state representation that integrates global topology information, existing scheduling mechanisms cannot fully consider the coupling relationships between nodes in the link when making decisions. This makes path selection and parameter configuration largely dependent on preset static rules or heuristic strategies. In addition, existing orchestration frameworks accumulate a large amount of runtime trajectory data during execution, but this is mostly limited to basic performance monitoring and alarm purposes. There is a lack of analysis mechanisms to automatically backtrack and locate problem nodes and problematic scheduling behaviors from low-quality trajectories. There is also a lack of standardized processes for parallel comparison and verification of configuration changes in isolated environments. As a result, the continuous iterative optimization of workflow execution strategies lacks systematic closed-loop support. Summary of the Invention

[0004] To address the shortcomings mentioned in the background technology, we propose an intelligent agent development system and method based on visual orchestration and large model collaboration.

[0005] The technical solution mainly consists of an agent development method based on visual orchestration and large-scale model collaboration, including the following steps:

[0006] Visual orchestration: The system provides a graphical user interface based on a directed acyclic graph, which allows users to select large model nodes, knowledge base retrieval nodes, code interpreter nodes, application interface request nodes and logical judgment nodes by dragging and dropping to build intelligent agent workflows. Data flow is defined between nodes through connections, and the output of the previous node is automatically mapped to the input variables of the next node. After orchestration is completed, the visual workflow is converted into a structured configuration file.

[0007] Graph State Awareness: Receives a structured configuration file, parses the workflow into a weighted directed graph, encodes the features of nodes and edges through a graph attention network, extracts node configuration parameters and runtime states to generate node feature vectors, extracts data flow direction and transmission volume to generate edge feature vectors, and generates a graph-level state vector containing global topology awareness through multi-layer message passing, quantifying the cascading coupling relationship between nodes;

[0008] Reinforcement learning dynamic decision-making: Based on node features and graph-level state vectors, extract the node execution state information of each candidate path, evaluate the scheduling priority of each candidate path, and select the optimal path as the actual execution path of the current workflow;

[0009] Experience replay and reflection analysis: After each round of workflow execution is completed, the complete execution trajectory is collected and stored in the experience replay buffer. Low-quality trajectories are filtered out and a large language model is called for in-depth analysis to locate problem nodes and generate parameter optimization suggestions. The optimization suggestions are associated with the corresponding trajectories and stored in the optimization knowledge base.

[0010] Online strategy update: Extract optimization suggestions from the optimization knowledge base, run multiple rounds of testing in parallel with the current solution in an isolated environment, determine the effectiveness of the optimization suggestions based on the test results, and formally write the effective optimization suggestions into the workflow configuration to achieve automatic iteration at the configuration level.

[0011] Preferably, the graph state perception step specifically includes:

[0012] Each node in the directed acyclic graph of the workflow is traversed, and the number of edges pointing from a node to its downstream nodes is counted as the out-degree value. The cumulative amount of data transmitted from a node to its direct downstream nodes and the historical average response time are collected from the node's operation log. The average response time is converted into a response adjustment coefficient through a nonlinear mapping. The cumulative data transmission volume is adjusted by the response adjustment coefficient to obtain the effective transmission volume. The effective transmission volumes of all downstream nodes are summarized and combined with the out-degree value for comprehensive processing to obtain the influence propagation degree of the node. The larger the influence propagation degree, the more critical the node's topological position. The influence propagation degree of each node is used as the initial attention weight of the graph attention network. Global topology-aware encoding is performed through multi-layer message passing to generate a graph-level state vector that integrates the topological coupling relationship of the entire graph.

[0013] Preferably, the reinforcement learning dynamic decision-making step specifically includes:

[0014] The system identifies nodes with multiple candidate paths as branch initiation nodes, uses the influence propagation degree of these nodes as the path baseline weight, traverses all nodes on each candidate path, collects the ratio of the number of successful executions of each node to the total number of executions as the single node execution success rate, calculates the average execution success rate of all nodes on the path as the path average execution success rate, obtains the highest and average values ​​of the path average execution success rates among all candidate paths, combines the path average execution success rate with the highest and average values ​​to obtain the path relative stability coefficient, counts the number of nodes in each candidate path and performs nonlinear compression to obtain the path length gain, combines the path baseline weight, path relative stability coefficient and path length gain to obtain the dynamic adjustability score of each candidate path, and selects the candidate path with the highest score as the actual execution path of the current workflow.

[0015] After each round of workflow execution is completed, data such as task completion quality score, end-to-end execution latency, semantic unit consumption, and execution success rate are collected. The execution latency and semantic unit consumption are adjusted with penalty coefficients and then combined with the quality score and execution success rate to obtain a reward value. The reward value is associated with all scheduling actions in this round to provide a trajectory screening basis for subsequent experience playback and reflection analysis steps.

[0016] Preferably, the experience playback and reflection analysis steps specifically include:

[0017] After each round of workflow execution is completed, the system automatically collects the input and output data of each node, the scheduling actions and corresponding reward values ​​output by the reinforcement learning decision module, and the explicit feedback marks made by the user on the final result. All data is spliced ​​into a complete execution trajectory and stored in the experience replay buffer. Based on the statistical distribution of historical trajectory reward values, a preset threshold is determined, and low-quality trajectories with reward values ​​below the preset threshold are periodically filtered out. Low-quality trajectories are sent to the large language model reflection analyzer. The reflection analyzer traces back the execution process of the trajectory segment by segment, locates the problem nodes and problem scheduling actions that cause quality degradation, generates parameter adjustment and optimization suggestions, binds the optimization suggestions to the corresponding trajectories and stores them in the optimization knowledge base, and provides candidate optimization solutions for the online strategy update steps.

[0018] Preferably, when a new optimization suggestion is added to the optimization knowledge base, an online strategy update is triggered. The specific steps of the online strategy update are as follows:

[0019] Starting from the dynamic adjustability score of the optimal path obtained in the reinforcement learning dynamic decision-making process, the dynamic adjustability scores of all candidate paths are statistically analyzed and the average score is calculated. The adjustability weight ratio of the optimal path is obtained by comprehensively analyzing the optimal path score and the average score of all paths. In an isolated environment, the optimized proposal and the current proposal are executed in parallel for the same number of rounds. After each round of task execution, the output quality is scored by the large language model evaluator. The average scores of the two proposals are calculated and compared to obtain the quality improvement ratio. The adjustability weight ratio and the quality improvement ratio are combined to obtain the configuration iteration adoption index. When the adoption index is greater than zero, the optimized proposal is written into the workflow configuration. When the adoption index is not greater than zero, the current configuration is retained and the optimized proposal is discarded.

[0020] This invention also provides an intelligent agent development system based on visual orchestration and large model collaboration, including a visual orchestration layer, a graph state perception module, a reinforcement learning decision module, an experience playback and reflection analysis module, and an online policy update module;

[0021] The visual orchestration layer is configured to provide a graphical user interface based on a directed acyclic graph, allowing users to drag and drop to build intelligent agent workflows. Data flow is defined between nodes through connections. After orchestration, the workflow is converted into a structured configuration file and sent to the graph state awareness module.

[0022] The graph state awareness module is configured to receive a structured configuration file and parse it into a weighted directed graph. It extracts the topological attributes and runtime state of each node through a graph attention network, calculates the influence propagation index of each node, and generates a graph-level state vector, which is then output to the reinforcement learning decision module.

[0023] The reinforcement learning decision module is configured to receive the influence propagation index and graph-level state vector, prioritize each candidate path and perceive the global topological coupling state, generate four types of scheduling actions (path selection, model routing, parameter adjustment, and resource allocation) and send them to the core engine layer for execution, while outputting the running data to the experience playback and reflection analysis module.

[0024] The experience replay and reflection analysis module is configured to receive runtime data and user display feedback markers, splice them into a complete execution trajectory and store it in the experience replay buffer, filter low-quality trajectories and call a large language model for backtracking analysis, locate problem nodes and generate optimization suggestions, and bind the corresponding trajectory to the optimization knowledge base.

[0025] The online strategy update module is configured to extract candidate optimization solutions from the optimization knowledge base, execute multiple rounds of tasks in parallel with the current configuration in an isolated test environment, collect and output quality score data for comparative analysis, and write the effective solutions into the workflow configuration based on the judgment results and synchronously update the component parameters of the visual orchestration layer to achieve closed-loop iteration at the configuration level.

[0026] Preferably, the graph state perception module includes a topology parsing unit, a feature extraction and influence calculation unit, a graph attention encoding unit, and a graph-level state generation unit;

[0027] The topology parsing unit is configured to receive structured configuration files from the visualization orchestration layer and parse the workflow into a weighted directed graph topology.

[0028] The feature extraction and impact calculation unit is configured to construct a feature vector containing configuration parameters and runtime state for each node, construct a feature vector containing data flow dependency and cumulative data transmission scale for each directed edge, and calculate the impact propagation index of each node based on the out-degree value, data transmission scale and average response time of each node, and use the impact propagation index as the initial attention weight of the graph attention encoding unit.

[0029] The graph attention encoding unit is configured to aggregate and encode the node feature vector and edge feature vector starting from the initial attention weight through the message passing mechanism of the multi-layer graph attention network, so that the representation vector of each node is fused with the topological association information of its neighboring nodes.

[0030] The graph-level state generation unit is configured to perform global pooling on the encoded representations of all nodes to generate a graph-level state vector containing the topological coupling relationships of the entire workflow. The influence propagation degree and the graph-level state vector are then output to the reinforcement learning decision module.

[0031] Preferably, the reinforcement learning decision module includes a state input unit, a policy network unit, a multi-dimensional action output unit, and a reward calculation unit;

[0032] The state input unit is configured to receive influence propagation degree and graph-level state vector, using influence propagation degree as input data for path tunability calculation and graph-level state vector as input state representation for policy network unit.

[0033] The policy network unit is configured to construct a decision network based on the proximal policy optimization algorithm, taking the graph-level state vector as input and outputting the probability distribution of path selection action, model routing action, parameter adjustment action and resource allocation action at each decision time step;

[0034] The multi-dimensional action output unit is configured to sample based on the probability distribution output by the policy network unit to determine the optimal execution path, the large language model used by the target node, the key parameter values ​​of the node to be executed, and the concurrency and timeout threshold of the target node.

[0035] The reward calculation unit is configured to collect task completion quality score, end-to-end execution latency, semantic unit consumption, and execution success rate data after each round of workflow execution. The execution latency and semantic unit consumption are converted into latency penalty coefficient and consumption penalty coefficient through nonlinear mapping, respectively. The quality score is combined with the two penalty coefficients to obtain a quality discount value. The execution success rate is multiplied with the quality discount value to obtain a reward value. The reward value is associated with all scheduling actions in this round and output to the experience playback and reflection analysis module.

[0036] Preferably, the experience playback and reflection analysis module includes a trajectory acquisition unit, an experience playback buffer, a low-quality trajectory filtering unit, a reflection analysis unit, and an optimized knowledge storage unit;

[0037] The trajectory acquisition unit is configured to collect complete execution trajectory data after each round of workflow execution, including the input and output content of each node, scheduling actions and corresponding reward values, and user display feedback markers;

[0038] An experience replay buffer is configured to store all execution trajectory data, providing a data source for short-term online fine-tuning and reflection analysis.

[0039] The low-quality trajectory screening unit is configured to periodically scan the experience playback buffer, screen low-quality trajectories with reward values ​​below a preset threshold, and output the screening results to the reflection analysis unit.

[0040] Among them, the preset threshold is dynamically determined based on the statistical distribution of all trajectory reward values ​​in the experience playback buffer;

[0041] The reflection analysis unit is configured to receive low-quality trajectories, call a large language model for in-depth analysis, locate problem nodes and problem scheduling actions, generate parameter optimization suggestions, and associate them with the corresponding trajectories and store them in the optimization knowledge storage unit.

[0042] The optimized knowledge storage unit is configured to store optimization suggestions and their associated trajectories in the optimization knowledge base, providing candidate optimization solutions for the online strategy update module.

[0043] Preferably, the online policy update module includes a short-term update unit, a long-term update unit, an isolated test environment, and a configuration writing unit;

[0044] The short-term update unit is configured to extract the latest trajectory data from the experience replay buffer after each preset round of workflow execution, and to fine-tune the policy network of the reinforcement learning decision module online so that the orchestration strategy can adapt to changes in the recent task distribution.

[0045] The long-term update unit is configured to extract optimization suggestions from the optimized knowledge storage unit, send them to the isolated test environment for effect verification, calculate the configuration iteration adoption index, and determine whether to execute configuration writing based on the adoption index.

[0046] The isolated test environment is configured to execute the optimized proposed solution and the current solution in parallel for the same number of rounds, calculate the average output quality score of the two solutions, calculate the quality improvement ratio, and return the results to the long-term update unit.

[0047] The configuration write unit is configured to receive adoption instructions from the long-term update unit, formally write the optimization suggestions into the workflow configuration, and synchronously update the configuration parameters of the corresponding components in the visual orchestration layer.

[0048] Compared with the prior art, the beneficial effects of the present invention are:

[0049] In this invention, the construction process of intelligent agent workflow is transformed from code writing to node configuration and connection combination through a graphical interface, which greatly reduces the development threshold. At the same time, a graph attention network is introduced to deeply encode the workflow topology, enabling the system to perceive the cascading coupling relationship between nodes. This breaks through the technical limitation of traditional static orchestration, which cannot perceive the upstream and downstream influences. On this basis, a reinforcement learning mechanism is used to achieve multi-dimensional scheduling and collaborative optimization of path selection, model routing, parameter adjustment and resource allocation. This ensures that the system can dynamically adjust the execution strategy according to the real-time task characteristics, comprehensively consider the multi-dimensional execution effect such as quality, efficiency, cost and stability, and significantly improve the orchestration adaptability and overall execution efficiency in complex long-link scenarios.

[0050] This invention integrates core functional modules such as orchestration and construction, topology resolution, model adaptation, and data storage into a unified architecture framework for collaborative operation. The modules are seamlessly connected through standardized data interfaces, enabling the workflow to form a complete integrated link from setup and execution to optimization.

[0051] This invention integrates the deep analysis capabilities of large language models into the optimization process. By backtracking low-quality execution trajectories segment by segment, it accurately locates problem nodes and scheduling behaviors, providing traceable decision-making basis for strategy optimization. Combined with retrieval-enhanced generation technology, it injects external knowledge into the model, effectively improving the knowledge blind spots in vertical domains. At the same time, it adopts a parallel comparative testing mechanism in an isolated environment to ensure that every configuration change is fully verified. High-quality execution trajectories are used for online fine-tuning of the policy network, enabling the orchestration strategy to continuously adapt to changes in task distribution. The close connection of each link forms a complete autonomous optimization system, allowing the intelligent agent workflow to continuously accumulate experience and complete self-upgrades during operation, achieving a dual improvement in development efficiency and operational quality. Attached Figure Description

[0052] Figure 1 This is a flowchart illustrating the overall operation of the present invention.

[0053] Figure 2 This is a flowchart of the modular workflow iterative optimization in this invention. Detailed Implementation

[0054] The present invention will now be described in further detail with reference to the accompanying drawings and preferred embodiments.

[0055] Example 1, refer to Figure 1-2 As shown, the agent development method based on visual orchestration and large model collaboration consists of five steps: visual orchestration, graph state awareness, reinforcement learning dynamic decision-making, experience replay and reflection analysis, and online policy update. Specifically:

[0056] First, the intelligent agent workflow is built through a graphical drag-and-drop interface, and the functional nodes are connected according to business logic and exported as a standardized topology description file.

[0057] Next, the file is analyzed in depth to extract the structured topology information of the workflow. The graph attention mechanism is used to perform multi-level feature fusion on node attributes and connection relationships to generate a topology state representation that reflects global dependencies. Then, based on this representation, each candidate execution branch is comprehensively evaluated to select the optimal execution path under the current task conditions.

[0058] During the actual operation of the workflow, the running status and execution results of each node are continuously collected, the abnormal trajectory is back-analyzed, the key nodes and scheduling behaviors that cause problems are accurately identified, and targeted optimization strategies are generated accordingly.

[0059] Finally, the generated optimization strategy is tested in a test environment isolated from the current configuration. Once it is confirmed to be effective, it is fixed into the workflow configuration to complete the automated upgrade of the system execution strategy.

[0060] In this embodiment, by introducing four mechanisms—topology awareness, dynamic decision-making, reflective analysis, and strategy evolution—between the traditional visual orchestration layer and the core execution layer, an intelligent workflow orchestration system with self-optimization capabilities is constructed. This system can continuously perceive the running status of the workflow, autonomously identify execution bottlenecks, and complete configuration optimization without human intervention, fundamentally solving the defects of path fixation, reliance on experience, and lack of topology awareness in the traditional static orchestration mode.

[0061] Example 2, refer to Figure 1-2 As shown, the graph state awareness step is the core link in realizing the quantification of workflow topology;

[0062] Specifically, the system first receives the structured configuration file generated by the visualization orchestration layer, parses the directed acyclic graph of the workflow into a weighted directed graph structure, then traverses each node in the graph, counts the number of edges from each node to downstream nodes as the out-degree value, collects the cumulative amount of data transmitted by each node to its direct downstream nodes from the node operation logs, collects the historical average response time of each node from the node operation logs, and then calculates the node's influence propagation degree to quantify the influence of each node on its downstream components. The specific calculation formula is as follows:

[0063] ;

[0064] In the formula, D i Let be the influence propagation degree of the i-th node, a comprehensive quantitative value of the node's influence on the downstream execution chain. `dout i` is the out-degree of node i, obtained by counting the number of directed edges from node i to downstream nodes, reflecting the direct influence range of node i. `Nout i` is the set of all direct downstream nodes of node i, extracted by traversing the directed graph structure. `j` is any downstream node in the set `Nout i`. ij The cumulative data volume transmitted from node i to node j is collected from the node operation log and measured in semantic units. It reflects the information supply from node i's output to downstream node j. (T) i The historical average response time of node i is obtained from the node's operation log and reflects the time consumption level of node i in processing tasks.

[0065] Furthermore, in the formula, the reciprocal of the out-degree... This is used to distribute influence equally among the number of downstream nodes. The response time is non-linearly mapped. When the response time is long, the term approaches 1, and the effective transmission volume is close to the original data volume. When the response time is short, the term increases, and the effective transmission volume is appropriately reduced, reflecting the characteristic that simple tasks with fast response have limited impact on downstream processes.

[0066] After calculation, D i The larger the value, the higher the criticality of the node in the workflow topology, and the more significant the impact of its state fluctuations on the downstream execution chain. This influences the D value of each node. i The values ​​are used as the initial attention weights of the graph attention network. Through a multi-layer message passing mechanism, global topology-aware encoding is completed, and finally a topology state representation that integrates the coupling relationships of the entire graph is generated.

[0067] In this embodiment, a quantitative formula for node influence is constructed by introducing three practically collectable operational metrics: out-degree, data transfer volume, and response time. This transforms the abstract workflow topology dependencies into calculable and comparable numerical indicators. The quantitative result provides a scientific basis for subsequent dynamic path selection, enabling the system to accurately identify key nodes in the workflow, prioritize the operational status and scheduling strategies of high-influence nodes, avoid investing excessive optimization resources in low-influence nodes, and improve the pertinence and effectiveness of the overall scheduling strategy. The quantitative result also serves as a unified data foundation for subsequent reinforcement learning decision-making, reflection analysis, and policy updates, ensuring that each step works collaboratively within the same topology cognitive framework.

[0068] Example 3, refer to Figure 1-2 As shown, the reinforcement learning dynamic decision-making process is the core step of selecting the optimal execution path based on topology perception. This step comprehensively considers the influence of the starting node of the candidate path, historical execution performance, and path structure characteristics, and quantifies and ranks the scheduling value of each candidate path.

[0069] Specifically, first, the node that generates a branch in the workflow is identified as the starting point of the path. The influence index of this node, calculated in the previous stage, is obtained as the basic weight of the path. Then, all nodes on each candidate path are traversed, and the historical execution success rate of each node is extracted. The average execution success rate of the entire path is then calculated. At the same time, the distribution of the average execution success rate of all candidate paths is statistically analyzed, and the maximum value and the overall mean are obtained. The number of nodes contained in each candidate path is also recorded. Based on the above data, the dynamic adjustability of the path is calculated to complete the quantitative evaluation. The specific calculation formula is as follows:

[0070] ;

[0071] In the formula, P k The dynamic adjustability score for the k-th candidate path is a comprehensive indicator that measures whether the path is suitable for priority selection in the current state. i The influence propagation degree of the starting branch node i of the path reflects the criticality of the starting point of the path in the entire workflow. k The average execution success rate of all nodes on path k is obtained by taking the arithmetic mean of the ratio of the historical successful execution counts to the total execution counts for each node on the path. This reflects the overall operational reliability of the path. max The highest average execution success rate among all candidate paths is determined by comparing the v values ​​of each candidate path. k The value obtained represents the optimal level of execution stability under the current conditions, v. p The average success rate of all candidate paths is calculated by applying v to all candidate paths. k The value is obtained by taking the arithmetic mean, reflecting the overall execution quality benchmark of the current workflow branch path, n k The total number of nodes contained in path k is obtained by statistically analyzing the path topology, reflecting the complexity of the path.

[0072] Furthermore, in the formula, This is the path relative stability coefficient. When the execution success rate of the candidate path is close to the highest level and the overall benchmark is high, this coefficient approaches the upper limit, indicating that the path has excellent stability.

[0073] ln(1+n) k The path length gain is used to nonlinearly compress the number of nodes, so that longer paths can obtain reasonable adjustable weights to preserve the optimization space.

[0074] P obtained by multiplying the three factors together k The higher the value, the more suitable the path is to be prioritized under the current conditions. The system selects the path with the highest score for execution based on this value, and at the same time combines the global topology state representation to complete joint scheduling optimization such as model routing, parameter adjustment and resource allocation.

[0075] Furthermore, after each round of workflow execution is completed, the system collects data on four indicators: task completion quality score, end-to-end execution latency, semantic unit consumption, and execution success rate. It then calculates a reward value to quantitatively evaluate the execution effect of that round. The specific calculation formula is as follows:

[0076] ;

[0077] In the formula, R is the reward value, which is a comprehensive quantitative evaluation index of the execution effect of a single round of workflow; Q is the task completion quality score, which is evaluated by the large language model evaluator on the final output, and the value ranges from 0 to 1. The higher the score, the better the output quality; α is the latency penalty coefficient, which is a preset constant used to adjust the penalty intensity of latency on the reward value; T is the end-to-end execution latency, collected from the execution log, in seconds, reflecting the overall response speed of the workflow from start to finish; S is the execution success rate, obtained by statistically analyzing the proportion of successfully completed nodes in this round of execution to the total number of nodes, and the value ranges from 0 to 1, reflecting the stability of execution; β is the consumption penalty coefficient, which is a preset constant used to adjust the penalty intensity of resource consumption on the reward value; and C is the semantic unit consumption, obtained from the execution log, in thousands of semantic units, reflecting the resource overhead of the language model called in this round of execution.

[0078] In the formula, the first term Dividing the quality score by the latency term adjusted for the penalty factor achieves a quality discount for high-latency executions. Divide the execution success rate by the consumption item adjusted by the penalty coefficient to achieve a stability discount for high-consumption executions. Multiply the two to get the reward value. The higher the value, the better the overall performance of this round of execution in the four dimensions of quality, efficiency, cost and stability.

[0079] The reward value R will serve as a key component of the execution trajectory. It will be combined with the input and output content of each node, the scheduling actions of this round of execution, and the explicit feedback markings made by the user on the final result to form a complete execution trajectory and stored in the experience replay buffer. The system will periodically scan the experience replay buffer and dynamically determine the screening threshold based on the statistical distribution of the reward values ​​of all trajectories in the buffer. Trajectories with reward values ​​lower than the threshold will be identified as low-quality trajectories and filtered out. These low-quality trajectories will be sent to the large language model reflection analyzer for in-depth backtracking analysis. The reflection analyzer will backtrack the execution process of the trajectory segment by segment, locate the problem nodes and problem scheduling actions that cause the quality degradation, generate targeted parameter adjustment and optimization suggestions, and bind the optimization suggestions with the full data of the corresponding trajectory and store them in the optimization knowledge base to provide candidate optimization schemes for subsequent configuration iteration adoption judgment.

[0080] High-quality trajectories with reward values ​​above the threshold are retained in the experience replay buffer as a training data source for online fine-tuning of the reinforcement learning policy network. This buffer is used to extract the latest trajectory data to optimize the policy network during short-term updates, so that the orchestration policy can continuously adapt to changes in recent task distribution.

[0081] In this embodiment, the path dynamic adjustability score integrates information from three dimensions—node influence, execution stability, and path structure—into a comprehensive index. The design of the relative stability coefficient allows for horizontal comparison of the execution quality of each candidate path on the same benchmark. The introduction of path length gain avoids the problem of short paths being preferentially selected due to their simple structure. The reward value formula discounts latency and consumption simultaneously through a double penalty mechanism, making the evaluation system take into account four dimensions: quality, efficiency, cost, and stability. This avoids suboptimal strategies caused by a single index. At the same time, the reward value-driven trajectory classification mechanism guides low-quality trajectories to deep reflective analysis driven by a large language model and guides high-quality trajectories to online fine-tuning training of the policy network, forming a complete data closed loop of execution, evaluation, classification, reflection, and optimization, ensuring that the experience of each execution can be fully explored and utilized.

[0082] Example 4, refer to Figure 1-2 As shown, the online policy update stage is the final execution stage for realizing the autonomous evolution of workflow configuration. This stage transforms the optimization suggestions accumulated in the previous stage into actual configuration change decisions, ensuring that each configuration update undergoes rigorous effect verification.

[0083] Specifically, firstly, the dynamic adjustability score of the optimal path selected in the previous step is obtained. Simultaneously, the dynamic adjustability scores of all candidate paths in the workflow are aggregated, and their overall average level is calculated. The optimal path score is compared with the overall average level to determine the relative scheduling value of that path within the entire workflow topology. At the same time, optimization suggestions to be verified are extracted from the optimization knowledge base. These suggestions are then tested in parallel with the current configuration in an independent, isolated environment for the same number of rounds. After each round, a large language model evaluator scores the output results for quality. The quality scores of the two sets of suggestions are aggregated, and their respective average levels are calculated. Based on the above data, the configuration iteration adoption index is calculated to complete the adoption decision. The specific calculation formula is as follows:

[0084] ;

[0085] in, ;

[0086] ;

[0087] In the formula, A is the configuration iteration adoption index, which is a comprehensive criterion for determining whether to write optimization suggestions into the workflow configuration, and P... k The dynamic adjustability score for the optimized path k reflects its scheduling optimization value in the current workflow. m is the total number of candidate paths in the workflow, obtained by traversing the workflow's branch structure. l is the index number of the candidate path. P lThe dynamic adjustability score for the l-th candidate path is given by q. new Output the average quality score for the new configuration scheme across n rounds of testing, where n is the total number of rounds in each test group, pre-set by the test scheme, and q old The old configuration outputs the average quality score across n rounds of testing;

[0088] In q new and q old In the calculation formula, q(new)u is the output quality score of the new configuration scheme in the u-th test round, which is evaluated round by round by the large language model evaluator, and q(old)u is the output quality score of the old configuration scheme in the u-th test round.

[0089] Furthermore, in the formula for calculating the iterative adoption index A, the first term of the formula... The adjustableness weight ratio of the optimized path. When the score of the path is significantly higher than the overall average, the weight ratio approaches the upper limit, indicating that the path has sufficient scheduling optimization space and is worth adjusting the configuration.

[0090] Formula afterword This represents the quality improvement rate of the new solution relative to the old solution. A positive value indicates that the new solution brings about quality improvement, while a negative value indicates a decrease in quality.

[0091] When the A value obtained by multiplying the two items is greater than zero, it means that the optimization suggestion applies to paths with sufficient optimization space and can actually improve the output quality. At this time, the optimization suggestion is formally written into the workflow configuration to complete the self-iteration. When the A value is not greater than zero, the current configuration is retained and the optimization suggestion is abandoned.

[0092] In this embodiment, the configuration iteration adoption index examines both the necessity and effectiveness of optimization through a two-factor judgment mechanism. The first factor ensures that the system only invests configuration change resources on paths with sufficient scheduling potential, avoiding ineffective iterations on low-value paths. The second factor ensures that the system only performs configuration updates when the quality has actually improved, avoiding quality regression caused by accidental fluctuations or misjudgments. This dual verification mechanism ensures that every configuration change is verifiable and traceable, enabling continuous autonomous evolution of workflow configuration while ensuring system stability. This allows the entire intelligent agent orchestration system to continuously accumulate experience, identify improvement directions, and complete self-upgrades during use, forming a stable and reliable automated closed-loop iteration capability.

[0093] Example 5, refer to Figure 1-2 As shown, this invention also provides an intelligent agent development system based on visual orchestration and large model collaboration, including a visual orchestration layer, a graph state perception module, a reinforcement learning decision-making module, an experience playback and reflection analysis module, and an online policy update module, specifically:

[0094] The visual orchestration layer is responsible for providing a graphical interface for workflow building, converting user-built business processes into structured description files and passing them down;

[0095] The graph state awareness module performs deep topology analysis on the file, extracts the operational characteristics and connection relationships of each node, calculates the influence index of each node on the downstream through the attention mechanism, and generates a topology state vector that reflects the global dependency relationship.

[0096] Based on the aforementioned influence indicators and topological state vectors, the reinforcement learning decision-making module comprehensively sorts all candidate execution branches in the workflow, selects the optimal execution path, and generates four types of scheduling instructions for path switching, model selection, parameter adjustment, and resource allocation for execution. It also synchronously pushes the data generated during the operation to the next stage.

[0097] The experience playback and reflection analysis module collects the operation records and user feedback of each node, assembles them into a complete execution file, filters out execution records with abnormal performance, calls the large language model to perform in-depth tracing of abnormal records, locates the specific link that caused the problem, forms reusable optimization strategies, and archives and stores them.

[0098] The online strategy update module extracts optimization strategies from the archive, compares their effects with the current strategy in an independent test environment, and after confirming the effectiveness of the optimization strategy, it officially deploys it to the workflow configuration and writes it back to the visual orchestration layer to maintain configuration consistency, thus completing the automatic upgrade of the system's execution capabilities.

[0099] In this embodiment, a complete data loop is formed among the five modules, encompassing data acquisition, topology perception, dynamic decision-making, experience accumulation, and strategy evolution. The graph state perception module enables the system to understand the inherent dependencies of the workflow. The reinforcement learning decision-making module achieves real-time collaborative optimization of multi-dimensional scheduling actions. The experience playback and reflection analysis module integrates the deep analysis capabilities of the large language model into the optimization process. The online strategy update module ensures that every configuration change is rigorously verified. The interconnected modules enable the entire system to autonomously accumulate experience, identify bottlenecks, and complete optimization upgrades during operation, significantly improving the adaptive capability and operational efficiency of the agent's workflow.

[0100] Meanwhile, the five modules are loosely coupled and integrated through standardized data interfaces. The topology description file output by the visualization orchestration layer is input to the graph state perception module. The topology state vector generated by the graph state perception provides a basis for reinforcement learning decisions. The execution data is synchronously pushed to the experience playback module to complete reflection and analysis. The optimized strategy is verified by the online policy update module and then written back to the visualization orchestration layer to maintain configuration consistency, ensuring the coordinated linkage and overall connection of the entire chain from orchestration construction to autonomous evolution.

[0101] It should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this invention should also be within the scope of protection of this invention.

Claims

1. An agent development method based on visual orchestration and large model collaboration, characterized by: Includes the following steps: Visual orchestration: The system provides a graphical user interface based on a directed acyclic graph, which allows users to select large model nodes, knowledge base retrieval nodes, code interpreter nodes, application interface request nodes and logical judgment nodes by dragging and dropping to build intelligent agent workflows. Data flow is defined between nodes through connections, and the output of the previous node is automatically mapped to the input variables of the next node. After orchestration is completed, the visual workflow is converted into a structured configuration file. Graph State Awareness: Receives a structured configuration file, parses the workflow into a weighted directed graph, encodes the features of nodes and edges through a graph attention network, extracts node configuration parameters and runtime states to generate node feature vectors, extracts data flow direction and transmission volume to generate edge feature vectors, and generates a graph-level state vector containing global topology awareness through multi-layer message passing, quantifying the cascading coupling relationship between nodes; Reinforcement learning dynamic decision-making: Based on node features and graph-level state vectors, extract the node execution state information of each candidate path, evaluate the scheduling priority of each candidate path, and select the optimal path as the actual execution path of the current workflow; Experience replay and reflection analysis: After each round of workflow execution is completed, the complete execution trajectory is collected and stored in the experience replay buffer. Low-quality trajectories are filtered out and a large language model is called for in-depth analysis to locate problem nodes and generate parameter optimization suggestions. The optimization suggestions are associated with the corresponding trajectories and stored in the optimization knowledge base. Online strategy update: Extract optimization suggestions from the optimization knowledge base, run multiple rounds of testing in parallel with the current solution in an isolated environment, determine the effectiveness of the optimization suggestions based on the test results, and formally write the effective optimization suggestions into the workflow configuration to achieve automatic iteration at the configuration level.

2. The agent development method based on visual orchestration and large model collaboration according to claim 1, characterized in that: The graph state perception step is specifically as follows: Each node in the directed acyclic graph of the workflow is traversed, and the number of edges pointing from a node to its downstream nodes is counted as the out-degree value. The cumulative amount of data transmitted from a node to its direct downstream nodes and the historical average response time are collected from the node's operation log. The average response time is converted into a response adjustment coefficient through a nonlinear mapping. The cumulative data transmission volume is adjusted by the response adjustment coefficient to obtain the effective transmission volume. The effective transmission volumes of all downstream nodes are summarized and combined with the out-degree value for comprehensive processing to obtain the influence propagation degree of the node. The larger the influence propagation degree, the more critical the node's topological position. The influence propagation degree of each node is used as the initial attention weight of the graph attention network. Global topology-aware encoding is performed through multi-layer message passing to generate a graph-level state vector that integrates the topological coupling relationship of the entire graph.

3. The agent development method based on visual orchestration and large model collaboration according to claim 2, characterized in that: The specific steps of the reinforcement learning dynamic decision-making process are as follows: The system identifies nodes with multiple candidate paths as branch initiation nodes, uses the influence propagation degree of these nodes as the path baseline weight, traverses all nodes on each candidate path, collects the ratio of the number of successful executions of each node to the total number of executions as the single node execution success rate, calculates the average execution success rate of all nodes on the path as the path average execution success rate, obtains the highest and average values ​​of the path average execution success rates among all candidate paths, combines the path average execution success rate with the highest and average values ​​to obtain the path relative stability coefficient, counts the number of nodes in each candidate path and performs nonlinear compression to obtain the path length gain, combines the path baseline weight, path relative stability coefficient and path length gain to obtain the dynamic adjustability score of each candidate path, and selects the candidate path with the highest score as the actual execution path of the current workflow. After each round of workflow execution is completed, data such as task completion quality score, end-to-end execution latency, semantic unit consumption, and execution success rate are collected. The execution latency and semantic unit consumption are adjusted with penalty coefficients and then combined with the quality score and execution success rate to obtain a reward value. The reward value is associated with all scheduling actions in this round to provide a trajectory screening basis for subsequent experience playback and reflection analysis steps.

4. The agent development method based on visual orchestration and large model collaboration according to claim 1, characterized in that: The specific steps of the experience playback and reflection analysis are as follows: After each round of workflow execution is completed, the system automatically collects the input and output data of each node, the scheduling actions and corresponding reward values ​​output by the reinforcement learning decision module, and the explicit feedback marks made by the user on the final result. All data is spliced ​​into a complete execution trajectory and stored in the experience replay buffer. Based on the statistical distribution of historical trajectory reward values, a preset threshold is determined, and low-quality trajectories with reward values ​​below the preset threshold are periodically filtered out. Low-quality trajectories are sent to the large language model reflection analyzer. The reflection analyzer traces back the execution process of the trajectory segment by segment, locates the problem nodes and problem scheduling actions that cause quality degradation, generates parameter adjustment and optimization suggestions, binds the optimization suggestions to the corresponding trajectories and stores them in the optimization knowledge base, and provides candidate optimization solutions for the online strategy update steps.

5. The agent development method based on visual orchestration and large model collaboration according to claim 3, characterized in that: When a new optimization suggestion is added to the knowledge base, an online policy update is triggered. The specific steps of the online policy update are as follows: Starting from the dynamic adjustability score of the optimal path obtained in the reinforcement learning dynamic decision-making process, the dynamic adjustability scores of all candidate paths are statistically analyzed and the average score is calculated. The adjustability weight ratio of the optimal path is obtained by comprehensively analyzing the optimal path score and the average score of all paths. In an isolated environment, the optimized proposal and the current proposal are executed in parallel for the same number of rounds. After each round of task execution, the output quality is scored by the large language model evaluator. The average scores of the two proposals are calculated and compared to obtain the quality improvement ratio. The adjustability weight ratio and the quality improvement ratio are combined to obtain the configuration iteration adoption index. When the adoption index is greater than zero, the optimized proposal is written into the workflow configuration. When the adoption index is not greater than zero, the current configuration is retained and the optimized proposal is discarded.

6. An agent development system based on visual orchestration and large model collaboration, based on the agent development method based on visual orchestration and large model collaboration as described in any one of claims 1-5, characterized in that: It includes a visualization orchestration layer, a graph state awareness module, a reinforcement learning decision-making module, an experience playback and reflection analysis module, and an online policy update module; The visual orchestration layer is configured to provide a graphical user interface based on a directed acyclic graph, allowing users to drag and drop to build intelligent agent workflows. Data flow is defined between nodes through connections. After orchestration, the workflow is converted into a structured configuration file and sent to the graph state awareness module. The graph state awareness module is configured to receive a structured configuration file and parse it into a weighted directed graph. It extracts the topological attributes and runtime state of each node through a graph attention network, calculates the influence propagation index of each node, and generates a graph-level state vector, which is then output to the reinforcement learning decision module. The reinforcement learning decision module is configured to receive the influence propagation index and graph-level state vector, prioritize each candidate path and perceive the global topological coupling state, generate four types of scheduling actions (path selection, model routing, parameter adjustment, and resource allocation) and send them to the core engine layer for execution, while outputting the running data to the experience playback and reflection analysis module. The experience replay and reflection analysis module is configured to receive runtime data and explicit user feedback tags, splice them into a complete execution trajectory and store it in the experience replay buffer, filter low-quality trajectories and call a large language model for backtracking analysis, locate problem nodes and generate optimization suggestions, and bind the corresponding trajectory to the optimization knowledge base; The online strategy update module is configured to extract candidate optimization solutions from the optimization knowledge base, execute multiple rounds of tasks in parallel with the current configuration in an isolated test environment, collect and output quality score data for comparative analysis, and write the effective solutions into the workflow configuration based on the judgment results and synchronously update the component parameters of the visual orchestration layer to achieve closed-loop iteration at the configuration level.

7. The intelligent agent development system based on visual orchestration and large model collaboration according to claim 6, characterized in that: The graph state perception module includes a topology parsing unit, a feature extraction and influence calculation unit, a graph attention encoding unit, and a graph-level state generation unit. The topology parsing unit is configured to receive structured configuration files from the visualization orchestration layer and parse the workflow into a weighted directed graph topology. The feature extraction and impact calculation unit is configured to construct a feature vector containing configuration parameters and runtime state for each node, construct a feature vector containing data flow dependency and cumulative data transmission scale for each directed edge, and calculate the impact propagation index of each node based on the out-degree value, data transmission scale and average response time of each node, and use the impact propagation index as the initial attention weight of the graph attention encoding unit. The graph attention encoding unit is configured to aggregate and encode the node feature vector and edge feature vector starting from the initial attention weight through the message passing mechanism of the multi-layer graph attention network, so that the representation vector of each node is fused with the topological association information of its neighboring nodes. The graph-level state generation unit is configured to perform global pooling on the encoded representations of all nodes to generate a graph-level state vector containing the topological coupling relationships of the entire workflow. The influence propagation degree and the graph-level state vector are then output to the reinforcement learning decision module.

8. The intelligent agent development system based on visual orchestration and large model collaboration according to claim 6, characterized in that: The reinforcement learning decision-making module includes a state input unit, a policy network unit, a multi-dimensional action output unit, and a reward calculation unit. The state input unit is configured to receive influence propagation degree and graph-level state vector, using influence propagation degree as input data for path tunability calculation and graph-level state vector as input state representation for policy network unit. The policy network unit is configured to construct a decision network based on the proximal policy optimization algorithm, taking the graph-level state vector as input and outputting the probability distribution of path selection action, model routing action, parameter adjustment action and resource allocation action at each decision time step; The multi-dimensional action output unit is configured to sample based on the probability distribution output by the policy network unit to determine the optimal execution path, the large language model used by the target node, the key parameter values ​​of the node to be executed, and the concurrency and timeout threshold of the target node. The reward calculation unit is configured to collect task completion quality score, end-to-end execution latency, semantic unit consumption, and execution success rate data after each round of workflow execution. The execution latency and semantic unit consumption are converted into latency penalty coefficient and consumption penalty coefficient through nonlinear mapping, respectively. The quality score is combined with the two penalty coefficients to obtain a quality discount value. The execution success rate is multiplied with the quality discount value to obtain a reward value. The reward value is associated with all scheduling actions in this round and output to the experience playback and reflection analysis module.

9. The intelligent agent development system based on visual orchestration and large model collaboration according to claim 8, characterized in that: The experience playback and reflection analysis module includes a trajectory acquisition unit, an experience playback buffer, a low-quality trajectory filtering unit, a reflection analysis unit, and an optimized knowledge storage unit; The trajectory acquisition unit is configured to collect complete execution trajectory data after each round of workflow execution, including the input and output content of each node, scheduling actions and corresponding reward values, and user display feedback markers; An experience replay buffer is configured to store all execution trajectory data, providing a data source for short-term online fine-tuning and reflection analysis. The low-quality trajectory screening unit is configured to periodically scan the experience playback buffer, screen low-quality trajectories with reward values ​​below a preset threshold, and output the screening results to the reflection analysis unit. Among them, the preset threshold is dynamically determined based on the statistical distribution of all trajectory reward values ​​in the experience playback buffer; The reflection analysis unit is configured to receive low-quality trajectories, call a large language model for in-depth analysis, locate problem nodes and problem scheduling actions, generate parameter optimization suggestions, and associate them with the corresponding trajectories and store them in the optimization knowledge storage unit. The optimized knowledge storage unit is configured to store optimization suggestions and their associated trajectories in the optimization knowledge base, providing candidate optimization solutions for the online strategy update module.

10. The intelligent agent development system based on visual orchestration and large model collaboration according to claim 9, characterized in that: The online policy update module includes a short-term update unit, a long-term update unit, an isolated test environment, and a configuration writing unit; The short-term update unit is configured to extract the latest trajectory data from the experience replay buffer after each preset round of workflow execution, and to fine-tune the policy network of the reinforcement learning decision module online so that the orchestration strategy can adapt to changes in the recent task distribution. The long-term update unit is configured to extract optimization suggestions from the optimized knowledge storage unit, send them to the isolated test environment for effect verification, calculate the configuration iteration adoption index, and determine whether to execute configuration writing based on the adoption index. The isolated test environment is configured to execute the optimized proposed solution and the current solution in parallel for the same number of rounds, calculate the average output quality score of the two solutions, calculate the quality improvement ratio, and return the results to the long-term update unit. The configuration write unit is configured to receive adoption instructions from the long-term update unit, formally write the optimization suggestions into the workflow configuration, and synchronously update the configuration parameters of the corresponding components in the visual orchestration layer.

Images