Structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning

Abstract

The invention discloses a structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning, and relates to the technical field of natural language processing, knowledge engineering and agent collaboration, and the method comprises the steps: receiving an original rule document, analyzing the document type, complexity and constraint conditions, and defining a task target and a success standard; and according to the task target, matching and scheduling the intelligent agent from the registered intelligent agent library, and further analyzing the capacity configuration of the intelligent agent for standby. Through the multi-agent cooperation and reinforcement learning technology, full-process automation of rule documents from input to structured analysis is realized, document types, complexity evaluation and constraint condition analysis can be automatically identified, and a clear task target and a success standard are generated; and the large language model generates a structured workflow according to task requirements and agent capabilities, so that the performability is ensured through logic verification, manual intervention is greatly reduced, and the processing efficiency and the system intelligence degree are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of natural language processing, knowledge engineering and agent collaboration, and particularly relates to a structured decision method based on multi-agent collaborative decision and reinforcement learning. BACKGROUND

[0002] In today's era of rapid information development, rule documents (including legal provisions, disability standards, technical specifications, etc.) widely exist in highly specialized fields such as medicine, law, finance, etc., and are important basis for enterprises and institutions to make decisions, assessments and services. These documents are not only the guiding principles of industry behavior, but also important basis for protecting rights and interests and standardizing operations. They are usually written in natural language, with complex structure, dense terminology, and interwoven logical relationships, and often involve multi-level conditional judgment and cross-referencing. However, with the explosive growth of information and the continuous improvement of rule complexity, traditional manual analysis and processing methods have been difficult to meet the needs of efficiency, accuracy and consistency. The rise of natural language processing technology has made it possible to automate the processing of rule documents.

[0003] Although there are document parsing methods based on natural language processing (NLP) or large language models in the prior art, there are still a series of technical bottlenecks. Traditional NLP models are unstable when dealing with long texts, professional terms and complex logic, and are prone to context breaks and semantic deviations. While large language models have strong language understanding capabilities, they have high randomness in output and inconsistent results when called multiple times, which seriously affects the credibility and usability of the parsing results. In addition, existing methods generally lack deep identification and verification mechanisms for the logical relationships between rules, and cannot effectively evaluate the quality of the parsing process and dynamically optimize it. Therefore, a structured decision method based on multi-agent collaborative decision and reinforcement learning is needed to solve the above technical problems and achieve efficient, accurate and automated structured decomposition of rule documents. SUMMARY

[0004] The present application aims to provide a structured decision method based on multi-agent collaborative decision and reinforcement learning to solve the problems raised in the background art.

[0005] To solve the above technical problems, the technical solution adopted by the present application is: The structured decision method based on multi-agent collaborative decision and reinforcement learning comprises the following steps: S1, receiving the original rule document, analyzing the document type, complexity and constraint conditions, and clarifying the task target and success criteria; S2, according to the task target, matching and scheduling agents from the registered agent library, and then analyzing the agent capability configuration for standby; S3, a pre-constructed large language model (LLM) generates a structured workflow sequence according to the task target and the agent capability, and performs logical feasibility verification; S4, each agent follows the workflow sequence, adjusts the agent allocation and the subtask execution sequence, and improves the execution efficiency and quality; S5, a comprehensive workflow quality scoring system is established for multi-dimensional quality evaluation, and a workflow quality score is calculated, wherein the workflow quality scoring system covers execution efficiency score, task quality score, collaboration effect score and maintainability score; S6, according to the reward signal generated by the multi-dimensional quality evaluation, the PPO algorithm is used to iteratively optimize the workflow generation strategy; S7, new collaboration modes are mined, existing modes are optimized, and efficient workflows and parameters are stored in the mode library as experience for subsequent task matching and optimization.

[0006] Further improvement of the technical scheme of the application is that S1 specifically includes: Receiving the original rule document input by the user, determining its category through a document type identification process, and evaluating the complexity of the document, including text length, structural complexity and domain (professional) term density; Analyzing the constraints in the document, including time limit, resource limit and key elements of quality requirement, to ensure comprehensive understanding and accurate control of task execution, and then based on the analysis of document type, document complexity and constraint condition, the specific target and success standard of the task are determined, and the basic framework of the workflow is preset.

[0007] Further improvement of the technical scheme of the application is that S2 specifically includes: Analyzing the task target, determining the structured analysis depth, knowledge extraction granularity and logical relationship restoration degree required by the task, and screening the intelligent agent candidate set with corresponding capabilities from the registered intelligent agent library, analyzing the intelligent agent capability configuration, and according to the preset capability evaluation matrix, quantitatively matching the candidate intelligent agents in the capability indexes including text understanding, entity recognition, logical reasoning and domain knowledge processing, and identifying the optimal intelligent agent combination; Combined with the document complexity and resource limit, the real-time state and historical performance of each intelligent agent are evaluated, dynamic scheduling decisions are made, intelligent agent task sequences are allocated according to the load balancing and collaboration efficiency principle, and exception handling and resource elasticity expansion capabilities are reserved; Complete intelligent agent work allocation and context initialization, transfer the scheduling results to the pre-constructed large language model (LLM), and all ready intelligent agents enter standby state.

[0008] Further improvement of the technical scheme of the application is that S3 specifically includes: The pre-constructed large language model (LLM) receives a task target, a constraint condition and an agent capability configuration as input, generates a structured workflow sequence based on a pre-trained workflow generation capability, defines a task step, covers an execution agent, a corresponding action, a natural language prompt instruction and a flow condition; The generated workflow sequence is subjected to logical consistency checking, including task dependency rationality, control structure effectiveness, data flow continuity and agent capability and task matching degree, to ensure that the workflow sequence meets executable requirements; Based on the verified workflow sequence, it is coded into an executable internal representation form, distributed to each ready agent, and initialized and prepared before execution, and enters the multi-agent collaborative execution phase.

[0009] The further improvement of the technical scheme of the application is that the process of generating a structured workflow sequence is: Based on the pre-constructed large language model, the input task target, constraint condition and agent capability configuration are structured and parsed, wherein the task target is disassembled into quantifiable sub-targets, the constraint condition is converted into formal rules, and the agent capability configuration is mapped into a skill graph; the large language model constructs a joint representation of task-constraint-capability based on the pre-trained context understanding capability, as the semantic alignment basis for workflow generation; Based on the parsed input context, the large language model calls the pre-trained workflow generation capability, automatically constructs a directed acyclic graph of task steps through graph generation technology, defines each step as a structured unit containing an execution agent, a corresponding action, a natural language prompt instruction and a flow condition, wherein the execution agent matches the optimal candidate from the capability configuration; the corresponding action generates executable operations based on the agent skill library; the natural language prompt instruction provides semantic guidance for action execution; the flow condition defines the dependency relationship between steps, and finally outputs a structured workflow sequence conforming to the predefined mode.

[0010] The further improvement of the technical scheme of the application is that the S4 specifically includes: Based on the task target and the constraint condition, the large language model analyzes the agent capability configuration, matches the optimal execution unit, disassembles the workflow sequence into parallel sub-task sequences, adjusts the agent resource allocation, and ensures task load balancing; The large language model reorders the sub-task execution sequence according to the input-output dependency relationship of each agent capability configuration, eliminates potential conflicts, generates a directed acyclic graph that meets the requirements of logical consistency and efficiency, and ensures the shortest critical path; During execution, the large language model monitors the quality of the agent output and the satisfaction of the constraints in real time, adjusts the task allocation (replaces inefficient agents) or optimizes the execution parameters through a dynamic feedback mechanism, and finally outputs a workflow sequence that meets the quality standards and is the most efficient.

[0011] The further improvement of the technical scheme of the application is that the S5 specifically comprises: Based on the four dimensions of execution efficiency, task quality, collaboration effect and maintainability, the sub-indicators and quantitative standards under each dimension are determined, and a comprehensive workflow quality scoring system is established, wherein the sub-indicators of execution efficiency include execution time, resource utilization and parallelism; the sub-indicators of task quality include task completion degree, result accuracy, consistency and integrity; the sub-indicators of collaboration effect include agent coordination, load balancing, communication efficiency and conflict resolution; the collaboration effect evaluation sub-task includes dependency rationality and parallel efficiency; the sub-indicators of maintainability include workflow complexity, modularization degree, error handling and scalability; the four dimensions of execution efficiency, task quality, collaboration effect and maintainability are allocated weights of 30%, 40%, 20% and 10% respectively to form a comprehensive scoring framework; The sub-indicators under each dimension are standardized and quantitatively scored, the scores of each dimension are calculated, and the scores of the four dimensions are weighted and summed according to the preset weights to obtain the workflow quality score, and a visual evaluation report is generated synchronously.

[0012] The further improvement of the technical scheme of the application is that the S6 specifically comprises: Based on the multi-dimensional quality evaluation results, a reward signal is generated, a reinforcement learning environment is constructed, and the workflow generation task is modeled as a Markov decision process, wherein the state is the current task context and system state, the action is the workflow structure decision, and the reward is converted from the workflow quality score; An optimization framework based on the PPO algorithm is built, the reward signal is integrated into the algorithm model, the pros and cons of the strategy are judged according to the reward signal feedback, the strategy parameters are adjusted, and the workflow generation strategy direction is explored; Through multiple rounds of iterative training, in each iteration, a workflow is generated according to the new strategy, a reward signal is obtained through quality evaluation, and the PPO algorithm is updated by feeding back the reward signal to the strategy, and the cycle is repeated until a workflow generation strategy that meets the requirements is generated.

[0013] The further improvement of the technical scheme of the application is that the process of exploring the workflow generation strategy direction is: Based on the PPO algorithm, an reinforcement learning optimization framework is built, the multi-dimensional quality evaluation is converted into a reward signal as the core input of the strategy optimization, and the strategy evaluation and update are carried out through the Actor-Critic network structure, wherein the Actor (strategy network) is a strategy generator, receives the current state information (such as task context, system resource state) and outputs the workflow generation decision, and the parameters thereof are optimized through the strategy gradient; the Critic (value network) is a value evaluator, predicts the long-term value based on the state-action pair, and provides a benchmark for strategy update; the reward signal directly reflects the advantages and disadvantages of the workflow generation strategy, and forms a closed-loop feedback. According to the reward signal feedback, the advantages and disadvantages of the current strategy are judged, and the robust updating of the strategy network parameters is carried out by using the clipping objective function of PPO, in each training batch, the probability ratio of the new and old strategies is calculated through importance sampling, the long-term income of the action is quantified by combining the generalized advantage estimator, and then the entropy regular term is introduced to constrain the output distribution of the Actor, the randomness of action selection is increased to encourage exploration, and the premature convergence to the local optimal strategy is avoided, the Critic is updated synchronously through the mean square error loss, and a precise value benchmark is provided for the strategy optimization, forming a stable training process of double-network collaborative optimization.

[0014] The further improvement of the technical scheme of the present application is that the S7 specifically comprises: The new cooperation mode is automatically mined from the successfully executed workflow instance, the efficient agent combination and workflow structure are identified, the execution trajectory is analyzed through the mode extraction algorithm, the reusable cooperation paradigm and key parameter configuration are refined, and the standardized mode description is formed; The existing cooperation mode is continuously optimized, the agent allocation strategy and workflow flow condition in the mode are adjusted based on the quality evaluation result and execution result data, the performance and application range of the mode are improved through parameter tuning and structure simplification, and the adaptability of the mode in different task scenarios is enhanced; The efficient workflow and parameter configuration verified as experience knowledge are stored in the mode library, and an indexing and retrieval mechanism is established, for subsequent new tasks, the adaptive cooperation mode is quickly recommended through mode matching and similarity calculation, the automatic generation and optimization of the workflow are supported, and the continuous accumulation and reuse of system experience are realized.

[0015] Due to the adoption of the above technical scheme, the present application has the following technical progress compared with the prior art: 1. The present application provides a structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning. Through multi-agent collaboration and reinforcement learning technology, the whole process automation from rule document input to structured analysis is realized. The method can automatically identify document type, assess complexity, analyze constraints, and generate clear task objectives and success criteria. Large language models generate structured workflows based on task requirements and agent capabilities, and then ensure executability through logical verification, significantly reducing manual intervention and improving processing efficiency and system intelligence.

[0016] 2. The present application provides a structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning. According to the task objective, the optimal agent combination is dynamically matched from the agent library. Real-time state and historical performance data are combined for resource allocation and task scheduling. Through load balancing and exception handling mechanisms, single agent overload is avoided, system robustness is improved, and dynamic adjustment and parallel processing are supported during workflow execution, significantly shortening task completion time, improving overall resource utilization and response speed.

[0017] 3. The present application provides a structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning. A comprehensive scoring system covering execution efficiency, task quality, collaboration effect and maintainability is established. The workflow is evaluated in real time through quantitative indicators. The reward signal based on quality score drives reinforcement learning optimization. PPO algorithm is used to iteratively improve workflow generation strategy, forming a closed-loop feedback mechanism of generation-execution-evaluation-optimization, continuously improving system output quality and stability. BRIEF DESCRIPTION OF DRAWINGS

[0018] To more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments described in the present application, and other drawings can also be obtained by those skilled in the art based on these drawings.

[0019] Figure 1 The workflow diagram of the structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning of the present application; Figure 2 The method flow diagram of the structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning of the present application. DETAILED DESCRIPTION

[0020] To make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0021] As shown in Embodiment 1, Figure 1 , Figure 2 The present application provides a structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning, including the following steps: S1, receiving the original rule document, parsing the document type, complexity and constraint conditions, clarifying the task target and success standard, receiving the original rule document input by the user, determining its category through the document type identification process, and evaluating the complexity of the document, including text length, structural complexity and domain (professional) term density, analyzing the constraint conditions in the document, covering time limit, resource limit and key elements of quality requirement, ensuring comprehensive understanding and accurate control of task execution, and then clarifying the specific target and success standard of the task based on the analysis of the document type, document complexity and constraint conditions, and presetting the basic framework of the work flow; The specific work content is: receiving the original rule document input by the user, starting the document type identification process, automatically determining the domain category to which the text belongs according to the text features and structure mode, such as law, medical treatment or technical specification, etc., quantitatively evaluating the complexity of the document in multiple dimensions, including text length (number of characters / paragraphs), structural complexity (chapter nesting level, cross-reference frequency) and domain (professional) term density (proportion of specific domain vocabulary), ensuring that the system can allocate resources and adjust processing strategies accordingly; on the basis of the complexity evaluation of the document, the constraint conditions embedded in the document are analyzed, covering time limit, resource limit and key elements of quality requirement, the time limit includes the overall processing time and the time limit of each sub-stage; the resource limit involves the maximum number of API calls, memory and computing power budget; the quality constraints include parsing accuracy, term consistency and logical integrity; through the fine extraction and formalized representation of the constraint conditions, a comprehensive understanding of the task execution environment is established; the analysis results of the document type, complexity and constraint conditions are integrated to clarify the specific target and success standard of the task, wherein the specific target of the task includes the depth of structured analysis, the granularity of knowledge extraction and the degree of logical relationship restoration; the success standard is refined into multi-dimensional indicators, including execution efficiency (processing time / resource consumption), result quality (accuracy / consistency) and collaboration effect (agent load balancing), according to which the work flow generation environment is initialized, and the basic framework of the work flow is preset; S2, according to the task target, match and dispatch the intelligent agent from the registered intelligent agent library, then analyze the intelligent agent capability configuration, standby, analyze the task target, clarify the structured analysis depth, knowledge extraction granularity and logical relationship restoration degree required by the task, and select the intelligent agent candidate set with corresponding ability from the registered intelligent agent library, analyze the intelligent agent capability configuration, according to the preset capability evaluation matrix, quantitatively match the candidate intelligent agent in the capability index including text understanding, entity recognition, logical reasoning and domain knowledge processing, identify the optimal intelligent agent combination, combine the document complexity and resource limitation, evaluate the real-time state and historical performance of each intelligent agent, execute dynamic scheduling decision, according to the load balancing and cooperation efficiency principle, allocate intelligent agent task sequence, and reserve exception handling and resource elastic expansion capability, complete intelligent agent work allocation and context initialization, deliver the scheduling result to the pre-constructed large language model (LLM), and all ready intelligent agents enter standby state; The specific work content is: systematically analyzing the task target, clarifying its specific requirements in the three dimensions of structured analysis depth, knowledge extraction granularity and logical relationship restoration degree, based on the registered intelligent agent library, screening out the candidate intelligent agent set with corresponding core capabilities (text understanding, entity recognition, logical reasoning and domain knowledge processing), according to the preset multi-dimensional capability evaluation matrix, quantitatively scoring and matching analysis of the performance of candidate intelligent agents in each capability index, identifying the optimal intelligent agent combination that can achieve the task target, wherein the capability index includes text understanding, entity recognition, logical reasoning and domain knowledge processing; After identifying the intelligent agent combination, combining the document complexity characteristics and resource limitation conditions, comprehensively evaluating the real-time running state, current load condition and historical performance data of each candidate intelligent agent, executing dynamic resource scheduling and task allocation decision, according to the load balancing and cooperation efficiency maximization principle, allocating clear task sequence and execution context for each intelligent agent, while reserving exception handling and resource elastic expansion capability to cope with sudden high load or intelligent agent failure during execution, ensuring the robustness and stability of the system under complex constraints; After completing the intelligent agent state evaluation and dynamic scheduling decision, enter the task allocation and context initialization stage, according to the scheduling result, allocate specific tasks to each intelligent agent and initialize its work context, including document content, task requirements and cooperation rules, ensure that the intelligent agent can accurately understand the task background and execution target, then deliver the scheduling result to the constructed large language model (LLM) as the key input of workflow generation, all ready intelligent agents enter standby state, ready to receive instructions and start task processing at any time; S3, a pre-constructed large language model (LLM) generates a structured workflow sequence according to the task target and the agent capability, and performs logical feasibility verification. The pre-constructed large language model (LLM) receives the task target, constraint condition and agent capability configuration as input, generates a structured workflow sequence based on the pre-trained workflow generation capability, defines the task steps, covers the execution agent, corresponding action, natural language prompt instruction and flow condition, performs logical consistency check on the generated workflow sequence, including task dependency rationality, control structure effectiveness, data flow continuity and agent capability and task matching degree, ensures that the workflow sequence meets the executable requirements, encodes the verified workflow sequence into an executable internal representation form, distributes it to each ready agent, completes the initialization preparation before execution, and enters the multi-agent collaboration execution phase; In addition, the process of generating a structured workflow sequence is: Based on the pre-constructed large language model, the input task target, constraint condition and agent capability configuration are structured and parsed. The task target is decomposed into quantifiable sub-targets (a structured sub-target list is obtained and associated with the overall task target), the constraint condition is converted into formal rules (covering time constraints and communication constraints), and the agent capability configuration is mapped into a skill graph (clearly defining the action type, parameter range and input / output format of each agent). Based on the pre-trained context understanding capability of the large language model, a joint representation of task-constraint-capability is constructed as the semantic alignment basis for workflow generation, providing a semantic alignment framework for subsequent workflow generation to ensure that the generation process meets the input requirements. Based on the parsed input context, the large language model calls the pre-trained workflow generation capability to automatically construct a directed acyclic graph of task steps through graph generation technology. Each step is defined as a structured unit containing an execution agent, a corresponding action, a natural language prompt instruction and a flow condition. The execution agent is matched with the optimal candidate from the capability configuration; the corresponding action generates executable operations based on the agent skill library; the natural language prompt instruction provides semantic guidance for action execution; and the flow condition defines the dependency relationship between steps. Finally, a structured workflow sequence conforming to the predefined mode is output. The specific work content is: a large language model (LLM) constructed in advance, according to the input task target, constraint condition and agent capability configuration, based on the pre-trained workflow generation capability, infers to automatically construct a structured workflow sequence, defines the detailed elements of each task step in a structured text form, including the executing agent, the corresponding action, the natural language prompt instruction and the flow condition between steps, wherein the task target defines the final achievement, the constraint condition sets the boundary and limitation in the execution process, the agent capability configuration describes the skill range of the callable agent, the task step is the link that needs to be experienced to achieve the target; the executing agent clearly indicates that each step is responsible for the corresponding agent; the corresponding action describes the operation of the agent in this step; the natural language prompt instruction provides guidance for the agent to execute the action; the flow condition specifies the conversion rule between steps; after constructing the workflow sequence, automatic execution logic consistency verification is performed to ensure its executability, the verification content includes whether the task dependency relationship is reasonable, whether the control flow structure is effective, whether the data transmission between different agents is coherent, and whether the actual capability of the agent matches the task requirements allocated, to identify and eliminate the logical conflicts, resource mismatches or structural defects in the workflow, ensure the logical rigor of the workflow and the actual running environment of the system, and then generate the structured workflow sequence required; based on the verified workflow sequence, it is converted into an internal representation form that can be parsed and scheduled, and dispatched to each ready agent, each agent receives its task description, execution context, input data and cooperation rules, completes the initialization before execution, enters the multi-agent cooperation execution phase, and each agent executes the task according to the sequence and condition defined by the workflow, through coordination communication and state synchronization, completes the structured analysis target of the rule document together; S4, each agent adjusts the agent allocation and subtask execution order according to the workflow sequence, improves the execution efficiency and quality, based on the task target and constraint condition, the large language model analyzes the capability configuration of each agent, matches the optimal execution unit, decomposes the workflow sequence into parallel subtask sequence, adjusts the agent resource allocation, ensures the task load balancing, the large language model reorders the subtask execution order according to the input and output dependency relationship of the agent capability configuration, combines the constraint condition, eliminates the potential conflict through graph algorithm, generates a directed acyclic graph that meets the requirements of logical consistency and efficiency, ensures the shortest key path, in the execution process, the large language model monitors the agent output quality and constraint satisfaction in real time, adjusts the task allocation (replaces the inefficient agent) or optimizes the execution parameters through the dynamic feedback mechanism, finally outputs the workflow sequence that meets the quality standard and has the optimal efficiency; The specific work involves: The large language model analyzes the agent's capability configuration based on the input task objectives and constraints, clarifying the supported action types, parameter ranges, and input / output formats for each agent. It then decomposes the workflow sequence into parallelizable sub-task sequences, assesses the resource consumption intensity of each sub-task sequence, and adjusts resource allocation ratios in real-time based on the agent's current load status to avoid performance bottlenecks caused by overload of a single agent. Finally, a load-balanced sub-task allocation scheme is generated. The large language model further analyzes the input / output dependencies between sub-tasks, constructing an initial directed acyclic graph (DAG), where nodes represent sub-tasks and edges represent data dependencies. To address constraints, the large language model uses a graph algorithm with topological sorting to rearrange the execution order of subtasks, eliminating potential conflicts. Simultaneously, by merging redundant operations or executing critical subtasks ahead of time, it shortens the overall execution time. The generated directed acyclic graph satisfies both logical consistency (no circular dependencies) and efficiency requirements (shortest critical path). During workflow execution, the large language model establishes a real-time monitoring mechanism to continuously track the output quality of each agent and the fulfillment of constraints. Through a dynamic feedback mechanism, it automatically identifies performance bottlenecks or substandard outputs and dynamically adjusts task allocation strategies accordingly, such as replacing inefficient agents or optimizing execution parameters, to achieve self-optimization of the workflow during operation. Ultimately, the output is a structured workflow sequence that achieves optimal quality and efficiency. S5. Establish a comprehensive workflow quality scoring system to conduct multi-dimensional quality assessment and calculate the workflow quality score. The workflow quality scoring system includes execution efficiency score (weight 30%), task quality score (weight 40%), collaboration effect score (weight 20%), and maintainability score (weight 10%). S6. Based on the reward signals generated by multi-dimensional quality assessment, use the PPO algorithm to iteratively optimize the workflow generation strategy; S7. Explore new collaboration models, optimize existing models, and store efficient workflows and parameters as experience in the model library for matching and optimization of subsequent tasks.

[0022] Example 2, as Figure 1 , Figure 2 As shown, based on Embodiment 1, the present invention provides a technical solution: Preferably, S5 specifically includes: Based on the four dimensions of execution efficiency, task quality, collaboration effect and maintainability, the sub-indicators and quantitative standards under each dimension are determined, and a comprehensive workflow quality scoring system is established. The sub-indicators of execution efficiency include execution time, resource utilization and parallelism. The sub-indicators of task quality include task completion, result accuracy, consistency and integrity. The sub-indicators of collaboration effect include agent coordination, load balancing, communication efficiency and conflict resolution. The sub-indicators of maintainability include workflow complexity, modularization degree, error handling and scalability. The four dimensions of execution efficiency, task quality, collaboration effect and maintainability are allocated weights of 30%, 40%, 20% and 10% respectively to form a comprehensive scoring framework. The sub-indicators under each dimension are standardized and quantitatively scored. The scores of each dimension are calculated, and the scores of the four dimensions are weighted and summed according to the preset weights to obtain the workflow quality score. A visual evaluation report is generated simultaneously. The specific work content is: based on the execution efficiency, task quality, collaboration effect and maintainability four dimensions, a comprehensive work flow quality scoring system is constructed, wherein the execution efficiency dimension covers execution time, resource utilization and parallelism, which is used to quantify the response speed and resource use efficiency of the work flow; the task quality dimension includes task completion degree, result accuracy, consistency and integrity, which is used to evaluate the reliability and completeness of the output results; the collaboration effect dimension involves agent coordination, load balancing, communication efficiency and conflict resolution capability, which reflects the coordination and stability performance in the process of multi-agent collaborative execution; the maintainability dimension includes work flow complexity, modularization degree, error handling mechanism and expansibility, which is used to measure the long-term operability and adaptability to changes of the work flow, the four dimensions are allocated according to the weight of 30%, 40%, 20% and 10%, forming a structured and quantifiable comprehensive scoring framework; the sub-indicators under each dimension are standardized and quantified, and the actual execution data is converted into a unified dimension score according to the preset benchmark and threshold, wherein in the execution efficiency, the execution time is calculated by the ratio of actual time consumption and preset threshold, the resource utilization is weighted average of CPU / memory occupation rate, and the parallelism is measured by the proportion of parallelizable subtasks; in the task quality, the completion degree is scored according to the task step execution rate, the accuracy is quantified by the error rate of the result and the benchmark data, the consistency is evaluated by the coincidence degree of the multi-agent output results, and the integrity is calculated by the proportion of indexes covering task requirements; in the collaboration effect, the coordination is scored by the rationality of agent task allocation, the load balancing is measured by the standard deviation of resource utilization, the communication efficiency is calculated by the message passing delay and throughput, and the conflict resolution capability is evaluated by the conflict occurrence frequency and solution time; in the maintainability, the work flow complexity is calculated by the number of work flow nodes and edges, the modularization degree is evaluated by the proportion of reusable subtasks, the error handling is scored by the fault detection and recovery time, and the expansibility is quantified by the impact range of new functions on the system; after the scores of each sub-indicator are normalized, the dimension scores are aggregated according to the dimension weight; the dimension scores are weighted and summed to obtain the final work flow quality score, the formula is: work flow quality score = execution efficiency score * 30% + task quality score * 40% + collaboration effect score * 20% + maintainability score * 10%, the score result adopts the percentage system, a visual evaluation report is generated simultaneously, including radar chart to show the dimension score distribution, column chart to compare the performance of sub-indicators, line chart to analyze the historical score trend, and key problem diagnosis and optimization suggestions are attached; S6 specifically comprises: Based on the multi-dimensional quality evaluation results, a reward signal is generated, a reinforcement learning environment is constructed, and the workflow generation task is modeled as a Markov decision process, wherein the state is the current task context and system state, the action is the workflow structure decision, the reward is converted from the workflow quality score, an optimization framework based on the PPO algorithm is built, the reward signal is integrated into the algorithm model, the pros and cons of the strategy are judged according to the reward signal feedback, the strategy parameters are adjusted, the workflow generation strategy direction is explored, and through multiple rounds of iterative training, in each iteration, a workflow is generated according to the new strategy, a reward signal is obtained through quality evaluation, and the reward signal is fed back to the PPO algorithm to update the strategy, and the cycle is repeated until a workflow generation strategy that meets the requirements is generated. In addition, the process of exploring the workflow generation strategy direction is: The PPO algorithm is used to build a reinforcement learning optimization framework. A multi-dimensional quality evaluation is converted into a reward signal as the core input for policy optimization. An Actor-Critic network structure is used for policy evaluation and updating. The Actor (policy network) is a policy generator that receives current state information (such as task context and system resource state) and outputs workflow generation decisions. Its parameters are optimized through policy gradient. The Critic (value network) is a value evaluator that predicts long-term value based on state-action pairs and provides a benchmark for policy updating. The reward signal directly reflects the pros and cons of the workflow generation strategy, forming a closed-loop feedback. In addition, the Actor and Critic share a state encoding layer to improve feature reuse efficiency. They complete decision generation and value evaluation through independent output layers, respectively, ensuring the decoupling of policy optimization and value estimation. According to the reward signal feedback, the pros and cons of the current strategy are judged. The clipping objective function of PPO is used for robust updating of the policy network parameters. In each training batch, the probability ratio of the new and old strategies is calculated through importance sampling. The long-term return of the action is quantified by the generalized advantage estimator, which avoids training shocks caused by excessive variance. The clipping mechanism limits the update range of the probability ratio to prevent performance collapse caused by excessive policy gradient step size. At the same time, it preserves enough exploration space. Then, the entropy regularization term is introduced to constrain the output distribution of the Actor. By increasing the randomness of action selection, exploration is encouraged, and premature convergence to a local optimal strategy is avoided. The Critic is updated synchronously through the mean square error loss to provide an accurate value benchmark for policy optimization, forming a stable training process of double-network collaborative optimization. Through a multi-level mechanism, exploration and utilization are balanced: At the policy generation level, the Actor combines deterministic decisions and random noise to strengthen exploration behavior in the early training stage and gradually transition to deterministic strategies for stable utilization of high-quality experience in the later stage. At the reward design level, intermediate rewards (immediate feedback of partial workflow fragments) and sparse rewards (final quality score of complete workflows) are combined to guide the strategy to balance between long-term goals and short-term exploration. At the parameter update level, the clipping coefficient and entropy regularization term weight are dynamically adjusted to adaptively control the exploration intensity according to the training stage. The Critic's value estimation results further guide the exploration direction. Through the advantage function, high-potential action regions are identified, enabling the strategy to evolve towards the global optimal workflow generation direction while maintaining stability. Specific work content is: based on multi-dimensional quality evaluation results, automatically generate the required reward signal of reinforcement learning, and build the corresponding reinforcement learning environment, the workflow generation task is modeled as a Markov decision process, where the state space includes the current task context information and the system real-time running state, the action space corresponds to the workflow structure generation and optimization decision, the reward function is converted from the workflow quality score to quantitatively reflect the pros and cons of the generation strategy; Build a reinforcement learning optimization framework based on the proximal policy optimization (PPO) algorithm, use the reward signal as the core basis for policy evaluation, judge the pros and cons of the current strategy according to the reward feedback, and continuously adjust the policy network parameters to balance exploration and utilization, gradually explore a better workflow structure generation direction, and realize the progressive optimization of the generation strategy; Through the process of multiple rounds of iterative training, the workflow generation strategy is gradually optimized, in each round of iteration, the workflow is generated according to the current strategy, the corresponding reward signal is obtained after actual execution and multi-dimensional quality evaluation, and it is fed back to the PPO algorithm to update the policy parameters, and the cycle continues, so that the strategy adjusts and improves itself based on historical performance until it converges to a satisfactory strategy that can stably generate high-quality workflows; S7 specifically includes: Automatically mine new collaboration patterns from successfully executed workflow instances, identify efficient agent combinations and workflow structures, and analyze execution trajectories through pattern extraction algorithms to extract reusable collaboration paradigms and key parameter configurations, form standardized pattern descriptions, continuously optimize existing collaboration patterns, adjust agent allocation strategies and workflow transfer conditions in the patterns based on quality evaluation results and execution result data, improve the performance and scope of the patterns through parameter tuning and structure simplification, enhance their adaptability in different task scenarios, store verified efficient workflow and parameter configurations as experience knowledge in the pattern library, and establish indexing and retrieval mechanisms. For subsequent new tasks, through pattern matching and similarity calculation, quickly recommend suitable collaboration patterns to support the automatic generation and optimization of workflows, and realize the continuous accumulation and reuse of system experience; Specific work content is: automatically mine new collaboration patterns from successfully executed workflow instances, identify efficient agent combinations and sequence structures by analyzing task execution traces, use pattern extraction algorithms to analyze the interaction process and data flow between agents, extract general collaboration paradigms and key parameter configurations, and form standardized pattern descriptions with clear structure and reusability; Perform continuous optimization mechanism on existing collaboration patterns, dynamically adjust agent allocation strategies and workflow step transfer conditions based on multi-dimensional quality evaluation results and historical execution data, optimize parameters and simplify structure to improve the performance and generalization ability of each mode, so that it can adapt to different complexity and domain characteristics of task scenarios; Store the proven efficient workflow and its parameter configuration as experience knowledge in the pattern library, and establish an efficient indexing and retrieval mechanism. When facing new tasks, the most suitable collaboration mode is recommended through semantic matching and similarity calculation, providing reliable priori for workflow generation, significantly improving the efficiency and quality of workflow generation, and forming a self-improving intelligent optimization closed loop.

[0023] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto. Any skilled person in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning, characterized in that, Includes the following steps: S1. Receive the original rule document, parse the document type, complexity and constraints, and clarify the task objectives and success criteria; S2. Based on the task objective, match and schedule agents from the registered agent library, and then analyze the agent's capability configuration to prepare for deployment. S3. A pre-built large language model generates a structured workflow sequence based on the task objective and the agent's capabilities, and performs logical feasibility verification. S4. Each agent follows the workflow sequence, adjusting agent allocation and subtask execution order; S5. Establish a comprehensive workflow quality scoring system to conduct multi-dimensional quality assessment and calculate workflow quality scores. The workflow quality scoring system includes execution efficiency scores, task quality scores, collaboration effectiveness scores, and maintainability scores. S6. Based on the reward signals generated by multi-dimensional quality assessment, use the PPO algorithm to iteratively optimize the workflow generation strategy; S7. Explore new collaboration models, optimize existing models, and store efficient workflows and parameters as experience in the model library.

2. The structured decision-making method based on multi-agent cooperative decision-making and reinforcement learning according to claim 1, characterized in that: S1 specifically includes: It receives raw rule documents input by users, determines their category through a document type identification process, and evaluates the complexity of the document, including text length, structural complexity, and domain terminology density. The constraints in the document are analyzed, covering key elements such as time limits, resource limitations, and quality requirements. Based on the analysis of document type, document complexity, and constraints, the specific goals and success criteria of the task are clarified, and the basic framework of the workflow is preset.

3. The structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning according to claim 2, characterized in that: S2 specifically includes: The task objectives are analyzed to clarify the required structured analysis depth, knowledge extraction granularity, and logical relationship restoration degree. A candidate set of intelligent agents with corresponding capabilities is selected from the registered intelligent agent library. The capability configuration of the intelligent agents is analyzed. Based on the preset capability evaluation matrix, the candidate intelligent agents are quantitatively matched in terms of capability indicators including text understanding, entity recognition, logical reasoning, and domain knowledge processing to identify the optimal combination of intelligent agents. Taking into account document complexity and resource constraints, the real-time status and historical performance of each agent are evaluated, dynamic scheduling decisions are made, and task sequences of agents are allocated according to the principles of load balancing and cooperation efficiency, while reserving the ability to handle exceptions and elastically expand resources. After completing the agent's work allocation and context initialization, the scheduling results are passed to the pre-built large language model, and all ready agents enter the standby state.

4. The structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning according to claim 1, characterized in that: S3 specifically includes: The pre-built large language model receives task objectives, constraints, and agent capability configurations as input. Based on the pre-trained workflow generation capability, it generates a structured workflow sequence, defines task steps, and covers the executing agent, corresponding actions, natural language prompts, and flow conditions. Perform logical consistency checks on the generated workflow sequence, including the rationality of task dependencies, the effectiveness of the control structure, the coherence of data flow, and the matching degree between the agent's capabilities and the tasks; Based on the validated workflow sequence, it is encoded into an executable internal representation, distributed to each ready agent, completes the initialization preparation before execution, and enters the multi-agent collaborative execution phase.

5. The structured decision-making method based on multi-agent collaborative decision-making and reinforcement learning according to claim 4, characterized in that: The process of generating the structured workflow sequence is as follows: Based on a pre-built large language model, the input task objectives, constraints, and agent capability configurations are structured and analyzed. The task objectives are decomposed into quantifiable sub-objectives, the constraints are transformed into formal rules, and the agent capability configurations are mapped to a skill graph. The large language model constructs a joint representation of task, constraint, and capability based on pre-trained contextual understanding capabilities. Based on the parsed input context, the large language model invokes its pre-trained workflow generation capabilities. Through graph generation technology, it automatically constructs a directed acyclic graph of task steps, defining each step as a structured unit containing an executing agent, a corresponding action, natural language prompts, and transition conditions. The executing agent matches the optimal candidate from the capability configuration; the corresponding action generates executable operations based on the agent's skill library; the natural language prompts provide semantic guidance for action execution; and the transition conditions define the dependencies between steps. Finally, it outputs a structured workflow sequence that conforms to a predefined pattern.

6. The structured decision-making method based on multi-agent cooperative decision-making and reinforcement learning according to claim 1, characterized in that: S4 specifically includes: Based on the task objectives and constraints, the large language model analyzes the capability configuration of each agent, matches the optimal execution unit, decomposes the workflow sequence into parallel sub-task sequences, and adjusts the agent resource allocation. Based on the input-output dependencies configured by the capabilities of each agent and combined with constraints, the large language model rearranges the execution order of subtasks using graph algorithms to generate a directed acyclic graph that meets the requirements of logical consistency and efficiency. During execution, the large language model monitors the output quality and constraint satisfaction of the agent in real time, and adjusts task allocation or optimizes execution parameters through a dynamic feedback mechanism, ultimately outputting a workflow sequence that meets quality standards and has the best efficiency.

7. The structured decision-making method based on multi-agent cooperative decision-making and reinforcement learning according to claim 6, characterized in that: S5 specifically includes: Based on four dimensions—execution efficiency, task quality, collaboration effectiveness, and maintainability—a comprehensive workflow quality scoring system is established, defining sub-indicators and quantitative standards for each dimension. Execution efficiency sub-indicators cover execution time, resource utilization, and parallelism; task quality sub-indicators include task completion, result accuracy, consistency, and completeness; collaboration effectiveness sub-indicators include agent coordination, load balancing, communication efficiency, and conflict resolution; collaboration effectiveness is evaluated by assessing the rationality of dependencies between subtasks and parallel efficiency; and maintainability sub-indicators cover workflow complexity, modularity, error handling, and scalability. The four dimensions—execution efficiency, task quality, collaboration effectiveness, and maintainability—are weighted at 30%, 40%, 20%, and 10% respectively to form the comprehensive scoring framework. Standardize and quantify the sub-indicators under each dimension, calculate the score for each dimension, and sum the scores of the four dimensions according to preset weights to obtain the workflow quality score, and generate a visual evaluation report simultaneously.

8. The structured decision-making method based on multi-agent cooperative decision-making and reinforcement learning according to claim 7, characterized in that: S6 specifically includes: Based on the multi-dimensional quality assessment results, a reward signal is generated, a reinforcement learning environment is constructed, and the workflow generation task is modeled as a Markov decision process, where the state is the current task context and system state, the action is the workflow structure decision, and the reward is transformed from the workflow quality score. Build an optimization framework based on the PPO algorithm, integrate reward signals into the algorithm model, judge the quality of the strategy based on the feedback of reward signals, adjust the strategy parameters, and explore the direction of workflow generation strategy; Through multiple rounds of iterative training, a workflow is generated based on the new strategy in each iteration. A reward signal is obtained through quality evaluation and fed back to the PPO algorithm to update the strategy. This process is repeated until a workflow generation strategy that meets the requirements is generated.

9. The structured decision-making method based on multi-agent cooperative decision-making and reinforcement learning according to claim 8, characterized in that: The process of exploring workflow generation strategy directions is as follows: A reinforcement learning optimization framework is built based on the PPO algorithm. The reward signal transformed from multi-dimensional quality assessment is used as the core input for policy optimization. The policy is evaluated and updated through an Actor-Critic network structure. In this structure, the Actor acts as a policy generator, receiving current state information and outputting workflow generation decisions. The Critic acts as a value evaluator, predicting long-term value based on state-action pairs, providing a benchmark for policy updates. The reward signal directly reflects the quality of the workflow generation policy, forming a closed-loop feedback. Based on the reward signal feedback, the quality of the current strategy is judged, and the PPO pruning objective function is used to robustly update the strategy network parameters. In each training batch, the probability ratio of the new and old strategies is calculated by importance sampling, and the long-term benefits of the action are quantified by combining generalized advantage estimation. Then, an entropy regularization term is introduced to constrain the output distribution of the Actor. By increasing the randomness of action selection, exploration is encouraged. The Critic is updated synchronously through mean squared error loss, forming a stable training process of dual-network collaborative optimization.

10. The structured decision-making method based on multi-agent cooperative decision-making and reinforcement learning according to claim 9, characterized in that: Specifically, S7 includes: It automatically mines new collaboration patterns from successfully executed workflow instances, identifies efficient combinations of intelligent agents and workflow structures, and analyzes execution trajectories through pattern extraction algorithms to extract reusable collaboration paradigms and key parameter configurations, forming standardized pattern descriptions. Continuously optimize the existing collaboration model, and adjust the agent allocation strategy and workflow conditions in the model based on quality assessment results and execution result data; Validated and effective efficient workflows and parameter configurations are stored as experiential knowledge in a pattern library, and an indexing and retrieval mechanism is established. For subsequent new tasks, suitable collaboration patterns are quickly recommended through pattern matching and similarity calculation.

Images