A task-based intelligent agent automatic construction method and system based on natural language dialogue

Abstract

This invention discloses an automatic construction method for task-oriented intelligent agents based on natural language dialogue. Step S1: Perform semantic analysis and feature extraction on each round of natural language dialogue to obtain structured semantic signals, and then construct a prompting stream based on these signals. Step S2: Quantitatively evaluate the execution effect of the current prompting stream based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, and calculate the comprehensive reward value. Step S3: Update the expected reward of only the current prompting stream in the same candidate prompting stream set using a multi-armed slot machine strategy. Step S4: Obtain the combined prompting stream execution sequence. Step S5: When the user explicitly confirms the current prompting stream execution sequence, freeze the current prompting stream execution sequence and generate a second intelligent agent definition. This invention enables non-professional users to generate reusable and evolvable task-oriented intelligent agents through natural language dialogue.

Description

Technical Field

[0001] This invention relates to a method for constructing intelligent agents for large language models. Background Technology

[0002] When users perform tasks using a Large Language Model (LLM), they typically use prompts, prompt flows, or agents. A prompt is the input text provided by the user to the LLM, usually containing instructions, background information, examples, questions, and context, with the aim of guiding the LLM to perform the task. A prompt flow not only contains prompts but also connects them, executable code, external tools, and the LLM to form an executable workflow. An agent is a software system capable of reasoning, planning, and executing tasks using the LLM. A prompt flow is a structured representation of a prompt strategy, falling between prompts and agents. A prompt flow typically consists of multiple prompt nodes connected in series, used to describe the calling order, conditional relationships, and parameter dependencies between these nodes.

[0003] The construction of intelligent agents in existing large language models is usually accomplished through methods such as prompt word writing, process arrangement, and parameter configuration. This is mainly aimed at professional users with certain engineering or AI (artificial intelligence) experience. Non-professional users find it difficult to directly participate in the construction of intelligent agents through natural language.

[0004] Existing methods of reusing intelligent agents typically involve copying existing agents entirely or modifying them partially. The design ideas, policy evolution process, and effectiveness of their internal prompts are often not recorded in a structured way, making it difficult to support continuous optimization based on historical interactions and feedback. Summary of the Invention

[0005] The technical problem to be solved by this invention is: how to make the construction of intelligent agents for large language models simpler, so that even non-professional users can build intelligent agents.

[0006] To address the aforementioned technical problems, this invention proposes an automatic construction method for task-oriented intelligent agents based on natural language dialogue, comprising the following steps: Step S1: Perform semantic analysis and feature extraction on each round of natural language dialogue between the user and the first intelligent agent to obtain structured semantic signals, and then construct a prompting stream based on the structured semantic signals. Step S2: Execute the predetermined task using the prompting stream constructed in each round of natural language dialogue. Quantitatively evaluate the execution effect of the current prompting stream based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, and calculate the comprehensive reward value R of the current prompting stream. tStep S3: Collect a set of cue flows with an update relationship as a candidate cue flow set; first, obtain the initial expected reward value for each cue flow in the same candidate cue flow set; then, treat the single cue flow constructed in this round of natural language dialogue as an optional action, based on its comprehensive reward value R. t A multi-armed slot machine strategy is used to update the expected reward value of the prompt stream, while the expected reward values ​​of the remaining prompt streams in the candidate prompt stream set remain unchanged. Step S4: For each candidate prompt stream set, only the prompt stream with the highest expected reward value is retained; prompt streams that do not constitute a candidate prompt stream set are retained as independent prompt streams; all retained prompt streams are sorted according to their expected reward values ​​from high to low to form a combined prompt stream execution sequence. Step S5: Steps S1 to S4 are repeated. When the user explicitly confirms that the effect of the current combined prompt stream execution sequence can be reused in the long term, the current combined prompt stream execution sequence is frozen, and a second agent definition is generated based on the combined prompt stream execution sequence.

[0007] Furthermore, in step S1, each round of natural language dialogue generates a local structured semantic signal, and all natural language dialogues form a global structured semantic signal; one or more rounds of natural language dialogue are used to generate a prompt stream, and each remaining round of natural language dialogue is used to update the prompt stream.

[0008] Furthermore, in step S1, the semantic analysis and feature extraction include: performing syntactic analysis and semantic understanding on the natural language dialogue text to identify semantic segments related to task intent, behavioral preferences, constraints, and feedback; and combining the current dialogue round's stage and historical dialogue context to perform role classification and semantic normalization processing on the identified semantic segments.

[0009] Further, in step S1, the structured semantic signals include: a task target signal, used to describe the specific task type, processing object, or target result that the user expects the large language model to complete; a user preference signal, used to characterize the user's preference for output format, reasoning style, decision-making tendency, or response granularity; a constraint signal, used to limit the range of behaviors and reasoning paths that the large language model can adopt during task execution; and a feedback signal, used to reflect the user's evaluation information on the output results of the large language model, including explicit evaluation instructions from the user and / or implicit feedback inferred from the user's interaction behavior patterns.

[0010] Furthermore, in step S1, the prompting flow is organized in a hierarchical structure, including at least system-level prompts for defining the roles and behavioral boundaries of the large language model, task-level prompts for describing the current task objective, strategy-level prompts for controlling the reasoning method and execution strategy, output-level prompts for defining the output format and style, and reflection or evaluation-level prompts for self-evaluation or correction of the generated results.

[0011] Furthermore, in step S2, the explicit feedback reward refers to collecting the user's confirmation, negation, or rating information on the output result of the first agent, and mapping it to a discrete reward value F. explicit Assign a user confidence coefficient c to different users. u This reflects the stability or professionalism of a user's historical behavior, thereby generating explicit reward signals with confidence. ; ;in, This is a feedback mapping function used to uniformly map different forms of explicit feedback to a standard reward range. The implicit behavioral reward refers to the number of times the user reuses the output of the first agent, the acceptance rate, and the usage duration score; the implicit reward signal... Where λ1, λ2, and λ3 are weighting coefficients; a time decay factor δ is introduced to decay the rewards for historical behavior; the implicit reward signal with time decay is used. Where t represents the current time, This indicates the time when the user's action occurred. The automatic reward evaluation refers to a comprehensive quality assessment of the task completion, consistency, and constraint satisfaction of the first agent's output, yielding a quality score Q(output); and further combining this with the risk penalty function g(Risk) to calculate the automatic reward signal. ; The overall revenue value R of the current prompt stream. t It is an explicit reward signal with confidence level. Implicit reward signals with time decay and automatic reward signal Generated after normalization; ; where α, β, and γ are weight parameters, and their values ​​are dynamically adjusted according to the overall running status of the current prompt stream.

[0012] Furthermore, in step S2, When the overall risk score Risk of the first agent's output is less than or equal to the first risk threshold τ1, it indicates that the current output of the first agent is within the acceptable safety range of the system, and no penalty is imposed on the quality score Q(output), with g(Risk) set to 1. When Risk > τ1 and < the second risk threshold τ2, it indicates that the current output of the first agent has a certain degree of potential risk or uncertainty, and the quality score Q(output) is nonlinearly suppressed through exponential decay; κ is a risk penalty intensity parameter used to adjust the steepness of the suppression. When Risk ≥ τ2, it indicates that the current output of the first agent has been judged to be high-risk or violates key constraints, and g(Risk) is set to 0, with the corresponding quality score Q(output) completely cleared to zero.

[0013] Furthermore, in step S3, as the multi-round natural language dialogue continues, the expected reward value of each prompt stream with an update relationship is continuously updated. The expected reward value of the prompt stream with better performance gradually increases, while the expected reward value of the prompt stream with worse performance gradually decreases, thereby achieving adaptive learning and optimization of the prompt streams with an update relationship.

[0014] Furthermore, in step S4, the second intelligent agent definition is a target intelligent agent generated by the large language model based on the natural language dialogue process. It is a solidified prompt stream, including policies, constraints, and preferences, and has the characteristics of being reusable and callable.

[0015] This invention also proposes an automatic construction system for task-oriented intelligent agents based on natural language dialogue, including a prompting flow construction module, a prompting flow evaluation module, a prompting flow selection module, a prompting flow combination module, and a prompting flow freezing module. The prompting flow construction module performs semantic analysis and feature extraction on each round of natural language dialogue between the user and the first intelligent agent to obtain structured semantic signals, and then constructs a prompting flow based on these structured semantic signals. The prompting flow evaluation module uses the prompting flow output from each round of natural language dialogue to execute a predetermined task, and quantitatively evaluates the execution effect of the current prompting flow based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, calculating the comprehensive reward value R of the current prompting flow. t The prompt flow selection module is used to select a set of prompt flows with an update relationship as a candidate prompt flow set; first, it obtains the initial expected benefit value of each prompt flow in the same candidate prompt flow set; then, it treats the single prompt flow constructed in this round of natural language dialogue as an optional action, based on its comprehensive benefit value R. tA multi-armed slot machine strategy is used to update the expected return value of the prompt stream, while the expected return values ​​of the remaining prompt streams in the candidate prompt stream set remain unchanged. The prompt stream combination module is used to retain only the prompt stream with the highest expected return value from each candidate prompt stream set; prompt streams that do not constitute a candidate prompt stream set are retained as independent prompt streams; all retained prompt streams are sorted according to their expected return values ​​from highest to lowest to form a combined prompt stream execution sequence. The prompt stream freezing module is used to freeze the current combined prompt stream execution sequence when the user explicitly confirms that its effect can be reused long-term, and to generate a second agent definition based on this combined prompt stream execution sequence.

[0016] The technical effect achieved by this invention is that it enables ordinary users (non-professional users) to gradually express their task goals and preferences through natural language dialogue, and introduces feedback and reward mechanisms in multiple rounds of interaction to automatically optimize and solidify the selection, order and structure of prompt words, thereby generating a reusable and evolvable task-oriented intelligent agent. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the automatic construction method for task-oriented intelligent agents based on natural language dialogue proposed in this invention.

[0018] Figure 2 This is a schematic diagram of the structure of the task-oriented intelligent agent automatic construction system based on natural language dialogue proposed in this invention.

[0019] The attached diagrams are labeled as follows: 1. Prompt stream construction module; 2. Prompt stream evaluation module; 3. Prompt stream selection module; 4. Prompt stream combination module; 5. Prompt stream freezing module. Detailed Implementation

[0020] Please see Figure 1 The automatic construction method for task-oriented intelligent agents (referred to as second intelligent agents) based on natural language dialogue proposed in this invention includes the following steps.

[0021] Step S1: The user and the first agent engage in one or more rounds of natural language dialogue. The first agent is based on a large language model and sits between the user and the model. It interacts with the user, carries the dialogue, and collects user intent and feedback information. Semantic analysis and feature extraction are performed on each round of natural language dialogue between the user and the first agent to obtain structured semantic signals. Based on these signals, a cue stream is constructed (generated or updated). For example, each round of natural language dialogue generates a local structured semantic signal, and all local structured semantic signals are aggregated to form a global structured semantic signal. One or several rounds of natural language dialogue are used to generate the cue stream, while the remaining rounds are used to update it—incrementally adjusting or structurally optimizing the cue stream. N rounds of natural language dialogue construct N cue streams, some of which may be related as initial and updated versions, or earlier and later updates. For example, the first round of natural language dialogue generates cue stream A1. The second round of natural language dialogue is used to update the cue stream A1, and the updated cue stream A1 is called cue stream A2. The third round of natural language dialogue is used to generate cue stream B1. The fourth round of natural language dialogue is used to update the cue stream A2, and the updated cue stream A2 is called cue stream A3.

[0022] The semantic analysis and feature extraction include: performing syntactic analysis and semantic understanding on natural language dialogue text to identify semantic fragments related to task intent, behavioral preferences, constraints, and feedback; and combining the current dialogue round's stage and historical dialogue context to perform role classification and semantic normalization on the identified semantic fragments.

[0023] The structured semantic signals include: (1) task target signals, which describe the specific task type, processing object or target result that the user expects the large language model to complete; (2) user preference signals, which characterize the user's preference for output form, reasoning style, decision tendency or response granularity; (3) constraint signals, which limit the range of behaviors and reasoning paths that the large language model can adopt in the process of performing the task; and (4) feedback signals, which reflect the user's evaluation information on the output result of the large language model, which may include the user's explicit evaluation instructions or implicit feedback inferred from the user's interaction behavior pattern.

[0024] The prompt flow is organized in a hierarchical structure, including at least system-level prompts to define the roles and behavioral boundaries of the large language model, task-level prompts to describe the current task objective, strategy-level prompts to control the reasoning method and execution strategy, output-level prompts to define the output format and style, and reflection or evaluation-level prompts for self-evaluation or correction of the generated results. This prompt flow exists as an intermediate representation within the system and is not directly displayed to the user, thus avoiding the need for the user to possess the ability to write or configure prompts.

[0025] Step S2: Execute the predetermined task using the prompting stream constructed from each round of natural language dialogue (referred to as the current prompting stream). The predetermined task refers to the task that the ultimately constructed task-oriented agent needs to perform. The effectiveness of the current prompting stream in executing the predetermined task is quantitatively evaluated based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, and the comprehensive reward value R of the current prompting stream is calculated. t Assuming N rounds of natural language dialogue generate N cue streams, this step calculates the N combined revenue values ​​R for these N cue streams. t .

[0026] The explicit feedback reward refers to collecting the user's confirmation, negation, or rating information on the output of the first agent (essentially the output of the large language model), and mapping it to a discrete reward value F. explicit Assign a user confidence coefficient c to different users. u This reflects the stability or professionalism of a user's historical behavior, thereby generating explicit reward signals with confidence. . .in, This is a feedback mapping function used to uniformly map different forms of explicit feedback to a standard reward range, thereby reducing the interference of low-reliability or sporadic feedback on policy learning.

[0027] The implicit behavioral reward refers to collecting implicit behavioral data such as the number of times a user reuses the output of the first agent, the acceptance rate, and the duration score. This implicit behavioral data is then modeled non-linearly to avoid reward inflation caused by simple linear superposition. Implicit reward signal. Here, λ1, λ2, and λ3 are weighting coefficients used to adjust the contribution of different implicit behavioral data to the overall reward. λ1≥0, λ2≥0, λ3≥0, and at least one weighting coefficient is greater than 0; λ1+λ2+λ3=1. A time decay factor δ is introduced to decay historical behavioral rewards, emphasizing the actual value of the cue stream in long-term stable use. Implicit reward signal with time decay. Where t represents the current time, It indicates the time point in time when a user action (such as accepting, reusing, continuing to use, or terminating the use of the output result of the first intelligent agent) occurs. Time decay is used to reduce the impact of short-term high-frequency operations on policy updates.

[0028] The automatic reward evaluation refers to a comprehensive quality assessment of the task completion, consistency, and constraint satisfaction of the first agent's output, resulting in a quality score Q(output); and further, an automatic reward signal is calculated by combining this with a risk penalty function g(Risk). This is to achieve joint modeling of quality and risk. Wherein, g(Risk) is used to nonlinearly suppress the quality score Q(output) based on the interval of the comprehensive risk score Risk of the first agent's output results.

[0029] As an example, .

[0030] When Risk ≤ the first risk threshold τ1, it means that the current output of the first agent is within the acceptable safety range of the system and no obvious compliance risks or quality hazards are detected. At this time, the output is considered to have positive value for the task objective, and no penalty is imposed on the quality score Q(output). The value of g(Risk) is 1.

[0031] When Risk > τ1 and < the second risk threshold τ2, it indicates that the current output of the first agent has a certain degree of potential risk or uncertainty. Although it has not yet constituted a clear violation or unacceptable problem, it has deviated from the ideal state in terms of compliance, reliability, or stability. For this type of medium-risk output, the quality score Q(output) is nonlinearly suppressed in an exponential decay manner. As the risk level increases, the automatic reward signal gradually weakens, thereby guiding the strategy to gradually move away from the high-risk output region while ensuring availability. κ is a risk penalty strength parameter used to adjust the steepness of the suppression. The larger κ is, the greater the degree of suppression and the steeper it is; conversely, the smaller κ is, the steeper it is.

[0032] When Risk ≥ τ2, it indicates that the current output of the first agent has been judged as high-risk or violates key constraints, such as triggering serious compliance rules, containing obvious factual errors, or posing potential security risks. For such outputs, g(Risk) is directly set to 0, and the corresponding quality score Q(output) is completely cleared. This prevents the corresponding cue stream from gaining positive benefits from this output during policy learning, thereby significantly reducing or even avoiding the generation of such high-risk results in subsequent optimizations.

[0033] The overall revenue value R of the current prompt stream t It is an explicit reward signal with confidence level. Implicit reward signals with time decay and automatic reward signal Generated after normalization. Here, α, β, and γ are weighting parameters, each ranging from 0 to 1, and α + β + γ = 1. The values ​​of α, β, and γ are dynamically adjusted based on the overall operational status of the current prompt flow (task context, historical execution results, user feedback behavior, and risk performance, etc.), thereby determining whether to emphasize user subjective feedback and user behavior signals or automatic assessment and risk control results at the current stage.

[0034] Step S3: When a cue stream is an updated version of another cue stream, these cue streams constitute a candidate cue stream set. For example, if cue stream A2 is an updated version of cue stream A1 and cue stream A3 is an updated version of cue stream A2, then cue streams A1, A2, and A3 constitute a candidate cue stream set. Based on the same principle, there may be zero, one, or more candidate cue stream sets, depending on the relationships between all the cue streams constructed from the entire natural language dialogue.

[0035] First, obtain the expected revenue E[R] for each cue stream in the same set of candidate cue streams. t Action i The initial value of the expected return (i.e., the selection probability). For a newly generated independent cue stream, its initial expected return can be a preset initial value or estimated based on historical statistics (e.g., a weighted average of historical returns). For an updated version of an existing cue stream, its initial expected return can inherit (copy) the expected return value of the original cue stream. For example, if the current round of natural language dialogue constructs cue stream A1, which is the first generated independent cue stream, its initial expected return uses a preset initial value. Similarly, if the current round of natural language dialogue constructs cue stream A2, which is an updated version of cue stream A1, then the initial expected return value of cue stream A2 inherits the expected return value of cue stream A1.

[0036] Subsequently, without updating the underlying large language model parameters, the single prompt stream constructed from the current round of natural language dialogue within the candidate prompt stream set is treated as an optional action. i Based on its comprehensive return value R tA multi-armed bandit (MAB) strategy is used to update the expected payoff of the prompt stream, while the expected payoffs of other prompt streams in the same candidate prompt stream set remain unchanged. For example, if the current round of natural language dialogue constructs prompt stream A2, which is an updated version of prompt stream A1, then the expected payoff value of prompt stream A2 is updated, while the expected payoff value of prompt stream A1 in the same candidate prompt stream set remains unchanged. Similarly, if the current round of natural language dialogue constructs prompt stream A3, which is an updated version of prompt stream A2, and prompt stream A2 is an updated version of prompt stream A1, then the expected payoff value of prompt stream A3 is updated, while the expected payoff values ​​of prompt streams A1 and A2 in the same candidate prompt stream set remain unchanged. The expected payoff is updated incrementally to gradually approximate the actual payoff distribution of the prompt stream.

[0037] For example, if the scheduled task is contract review, this round of natural language dialogue constructs cue flow A3, which is an updated version of cue flow A2, and cue flow A2 is an updated version of cue flow A1. Cue flows A1, A2, and A3 constitute a candidate cue flow set. The overall benefit value R of cue flow A3 is... t The value is 0.82, indicating that the expected return E[R] of flow A3 is... t If the initial value of |A3] is 0.75, then the incremental update method of the multi-armed slot machine is adopted to update the comprehensive reward value R of the prompt stream A3 constructed in this round of natural language dialogue. t The actual return is merged and updated with the initial value of the expected return (historical expected return). For example, the expected return value of cue stream A3 is updated to E[R] using a learning rate of 0.1. t |A3] = 0.75 + 0.1 × (0.82 − 0.75) = 0.757. This round of natural language dialogue did not construct cue flows A1 and A2, and their expected rewards remained unchanged. As multiple rounds of natural language dialogue continue, the expected rewards of the cue flows that have an update relationship with each other will be continuously adjusted. The expected rewards of the better-performing cue flows will gradually increase, while the expected rewards of the worse-performing cue flows will gradually decrease, thus achieving adaptive learning and optimization of cue flows that have an update relationship with each other.

[0038] Step S4: For each candidate suggestion stream set, only the suggestion stream with the highest expected return value is retained; the remaining suggestion streams do not participate in subsequent execution. Suggestion streams that do not constitute a candidate suggestion stream set are retained as independent suggestion streams. All retained suggestion streams are sorted from highest to lowest expected return value to form a combined suggestion stream execution sequence. The order of the suggestion streams is the execution order. During execution, suggestion streams ranked higher participate in task processing first, and their outputs can serve as input or context for subsequent suggestion streams, thus forming a multi-stage collaborative execution mechanism. This approach prioritizes high-quality suggestion streams while preserving multi-strategy collaborative capabilities to improve overall task processing efficiency.

[0039] For example, after a round of natural language dialogue, the first candidate cue stream set consists of cue streams A1, A2, and A3, with only cue stream A3, which has the highest expected return value, retained. The second candidate cue stream set consists of cue streams B1 and B2, with only cue stream B2, which has the highest expected return value, retained. Cue stream C1 does not constitute a candidate cue stream set and is retained as an independent cue stream. The retained cue streams A3, B2, and C1 are sorted from highest to lowest expected return value to obtain a combined cue stream execution sequence {A3, B2, C1}.

[0040] Step S5: Repeat steps S1 to S4. When the user explicitly confirms that the effect of the currently combined prompt stream execution sequence can be reused long-term, freeze (solidify) the currently combined prompt stream execution sequence, and generate a second agent definition based on the combined prompt stream execution sequence for saving and recall. The generated second agent definition is the independent task-oriented agent automatically constructed by this invention. The second agent is the target agent generated by the large language model based on the natural language dialogue process. It is a solidified prompt stream, typically including policies, constraints, and preferences, and has the characteristics of reusability and recallability.

[0041] In this invention, users do not need to directly interact with prompt words, prompt streams, or agent configuration details; they only need to express business rules and judgment logic through natural language dialogue. The large language model automatically parses the user's expression into task goals, preferences, and constraints in the background and maps them into a structured prompt stream. For the user, the objects of operation are always "rules" and "conclusions," rather than prompt text or execution flow, thereby significantly reducing the usage threshold for building and maintaining agents.

[0042] The following example, using a legal contract review (pre-defined task) scenario, illustrates this application. Legal personnel progressively describe the business rules for contract review through multi-round natural language dialogue with the first intelligent agent, such as the method for determining the authorization period, the differences in review for different types of contracts, and risk labeling standards. After each round of dialogue, the first intelligent agent constructs a corresponding prompt flow based on the current rule set and uses this prompt flow to review and output the contract text (execute the pre-defined task). As the dialogue progresses, the first intelligent agent gradually learns and solidifies the complete review rule system, ultimately forming a reusable legal contract review intelligent agent (the second intelligent agent).

[0043] During the construction of the cue stream, the first agent allows the user to supplement or revise the rules multiple times. For example, suppose a user inputs forty review rules in ten conversations, but in the eighth conversation, the description of a certain rule is inaccurate, leading to logical ambiguity. Suppose the eighth natural language conversation constructs cue stream X1. After this rule is introduced, the system evaluates its stability based on subsequent review results and provides hints or annotations for the relevant conclusions in the output. When the user discovers a problem with the rule and negates or corrects it through natural language, the first agent recognizes this feedback as a negative reward signal and automatically locates the cue stream X1 associated with this feedback. Upon receiving the negative reward, the first agent does not simply add a new rule to the existing cue stream X1, but updates cue stream X1 to cue stream X2, and reverts cue stream A2 to the stable version before the introduction of the problematic rule. Based on this, the relevant problematic rules are replaced or rewritten. At this point, cue streams X1 and X2 constitute a candidate cue stream set. Subsequently, the first agent regenerates the review logic and output based on the corrected cue flow X2, and provides feedback to the user on the rule updates and changes in cue flows X1 to X2. If cue flow X2 receives a positive reward signal, then cue flow X2 has a higher expected reward value in the same set of candidate cue flows, and only cue flow X2 is retained. Cue flow X1 is effectively discarded, and its probability of being selected into the combined cue flow execution sequence is infinitely reduced unless subsequent rule modifications introduce new cue flows with an update relationship. In this way, the first agent achieves controllable revocation and correction of erroneous rules, avoiding the problem of model context pollution.

[0044] Throughout the process, the user's evaluation of the first agent's output, their rule-correction actions, and their adoption of the generated results are all transformed into explicit or implicit reward signals by the system. These reward signals do not affect the underlying language model parameters but are directly used to optimize the combination strategy of the prompt flow, including the activation status of prompt words, their execution order, and structural organization. Through the reward-driven prompt word strategy optimization mechanism, the first agent can gradually converge to a stable second agent definition that meets the user's business needs.

[0045] Finally, the first agent freezes the validated and stable combination of prompt flow execution sequences, generating a complete second agent definition. This second agent definition not only includes the structured prompt flow execution sequence but also the corresponding execution strategy, long-term memory information, and reward trajectory records, enabling the second agent to be reusable, interpretable, and sustainably evolving. In subsequent use, users only need to invoke the generated second agent to complete contract review tasks without needing to redescribe or configure review rules.

[0046] Please see Figure 2 The automatic construction system for task-oriented intelligent agents (referred to as the second intelligent agent) based on natural language dialogue proposed in this invention includes a prompt flow construction module 1, a prompt flow evaluation module 2, a prompt flow selection module 3, a prompt flow combination module 4, and a prompt flow freezing module 5. Figure 2 The system shown corresponds to Figure 1 The method shown.

[0047] The prompting stream construction module 1 is used to perform semantic analysis and feature extraction on each round of natural language dialogue between the user and the first intelligent agent to obtain structured semantic signals, and then construct a prompting stream based on the structured semantic signals.

[0048] The prompting flow evaluation module 2 is used to execute a predetermined task using the prompting flow (referred to as the current prompting flow) constructed in each round of natural language dialogue. It quantitatively evaluates the execution effect of the current prompting flow based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, and calculates the comprehensive benefit value R of the current prompting flow. t .

[0049] The cue flow selection module 3 is used to form a candidate cue flow set when a cue flow is an updated version of another cue flow. First, the initial expected return of each cue flow in the same candidate cue flow set is calculated. Then, a multi-armed slot machine strategy is used to update the expected return of the single cue flow constructed from the current round of natural language dialogue within the candidate cue flow set.

[0050] The prompt flow combination module 4 is used to select only the prompt flow with the highest expected return value from each candidate prompt flow set. Prompt flows that do not constitute a candidate prompt flow set are retained as independent prompt flows. All retained prompt flows are sorted from highest to lowest expected return value to form a combined prompt flow execution sequence.

[0051] The prompt stream freezing module 5 is used to freeze the current combined prompt stream execution sequence when the user clearly confirms that the effect of the current combined prompt stream execution sequence can be reused for a long time, and to generate a second agent definition based on the combined prompt stream execution sequence.

[0052] Compared with existing technologies, this invention has the following beneficial effects. First, this invention significantly lowers the threshold for building intelligent agents in large language models, allowing even non-professional users to create complex intelligent agents. Second, this invention achieves automatic learning and structured management of prompt streams, improving the reusability and stability of the combined prompt stream execution sequence. Third, this invention supports versioned management and evolution of intelligent agents, forming accumulative and auditable intelligent agent assets, improving the stability and consistency of task output results. Fourth, this invention enhances the interpretability and controllability of the system, meeting the security and compliance requirements of enterprise-level applications.

[0053] The above are merely preferred embodiments of the present invention and are not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for automatically constructing task-oriented intelligent agents based on natural language dialogue, characterized in that, Includes the following steps; Step S1: Perform semantic analysis and feature extraction on each round of natural language dialogue between the user and the first intelligent agent to obtain structured semantic signals, and then construct a prompt stream based on the structured semantic signals; Step S2: Execute the predetermined task using the cue stream constructed from each round of natural language dialogue. Quantitatively evaluate the execution effect of the current cue stream based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, and calculate the comprehensive benefit value R of the current cue stream. t ; Step S3: Collect a set of cue flows with an update relationship as a candidate cue flow set; first, obtain the initial expected reward value for each cue flow in the same candidate cue flow set; then, treat the single cue flow constructed in this round of natural language dialogue as an optional action, based on its comprehensive reward value R. t The expected payoff value of the suggestion stream is updated using a multi-armed slot machine strategy, while the expected payoff values ​​of the remaining suggestion streams in the candidate suggestion stream set remain unchanged. Step S4: For each candidate suggestion stream set, retain only the suggestion stream with the highest expected return value; for suggestion streams that do not constitute a candidate suggestion stream set, retain them as independent suggestion streams; sort all retained suggestion streams according to their expected return value from high to low to form a combined suggestion stream execution sequence; Step S5: Repeat steps S1 to S4. When the user clearly confirms that the effect of the current combined prompt stream execution sequence can be reused for a long time, freeze the current combined prompt stream execution sequence and generate a second agent definition based on the combined prompt stream execution sequence.

2. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 1, characterized in that, In step S1, each round of natural language dialogue generates a local structured semantic signal, and all natural language dialogues form a global structured semantic signal; one or more rounds of natural language dialogue are used to generate a prompt stream, and each of the remaining rounds of natural language dialogue is used to update the prompt stream.

3. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 1, characterized in that, In step S1, the semantic analysis and feature extraction include: performing syntactic analysis and semantic understanding on the natural language dialogue text to identify semantic segments related to task intent, behavioral preferences, constraints, and feedback; and combining the current dialogue round's stage and historical dialogue context to perform role classification and semantic normalization on the identified semantic segments.

4. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 1, characterized in that, In step S1, the structured semantic signal includes: Task target signal is used to describe the specific task type, processing object, or target result that the user expects the large language model to complete; User preference signals are used to characterize a user's preference for output format, reasoning style, decision-making tendency, or response granularity. Constraint signals are used to limit the range of behaviors and inference paths that a large language model can adopt during task execution; Feedback signals are used to reflect the user's evaluation information on the output of the large language model, including the user's explicit evaluation instructions and / or implicit feedback inferred from the user's interaction behavior patterns.

5. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 1, characterized in that, In step S1, the prompting flow is organized in a hierarchical structure, including at least system-level prompts for defining the roles and behavioral boundaries of the large language model, task-level prompts for describing the current task objective, strategy-level prompts for controlling the reasoning method and execution strategy, output-level prompts for defining the output form and style, and reflection or evaluation-level prompts for self-evaluation or correction of the generated results.

6. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 1, characterized in that, In step S2, the explicit feedback reward refers to collecting the user's confirmation, negation, or rating information on the output of the first agent and mapping it to a discrete reward value F. explicit Assign a user confidence coefficient c to different users. u This reflects the stability or professionalism of a user's historical behavior, thereby generating explicit reward signals with confidence. ; ;in, This is a feedback mapping function used to uniformly map different forms of explicit feedback to a standard reward range; The implicit behavioral reward refers to the collection of user data on the reuse count, acceptance rate, and duration score of the first agent's output; the implicit reward signal. Where λ1, λ2, and λ3 are weighting coefficients; a time decay factor δ is introduced to decay the rewards for historical behavior; the implicit reward signal with time decay is used. Where t represents the current time, Indicates the time when the user's action occurred; The automatic reward evaluation refers to a comprehensive quality assessment of the task completion, consistency, and constraint satisfaction of the first agent's output, resulting in a quality score Q(output); and further, an automatic reward signal is calculated by combining this with a risk penalty function g(Risk). ; ; The overall revenue value R of the current prompt stream t It is an explicit reward signal with confidence level. Implicit reward signals with time decay and automatic reward signal Generated after normalization; ; where α, β, and γ are weight parameters, and their values ​​are dynamically adjusted according to the overall running status of the current prompt stream.

7. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 6, characterized in that, In step S2 ; When the comprehensive risk score Risk of the first agent's output is less than or equal to the first risk threshold τ1, it means that the current output of the first agent is within the acceptable safe range of the system, and no penalty is imposed on the quality score Q(output), and g(Risk) takes the value of 1. When Risk > τ1 and < the second risk threshold τ2, it indicates that the current output of the first agent has a certain degree of potential risk or uncertainty. The quality score Q(output) is nonlinearly suppressed by exponential decay. κ is a risk penalty intensity parameter used to adjust the steepness of the inhibition; When Risk≥τ2, it means that the current output of the first agent has been judged as high risk or violated the key constraints. The value of g(Risk) is set to 0, and the corresponding quality score Q(output) is completely cleared.

8. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 6, characterized in that, In step S3, as the multi-round natural language dialogue continues, the expected reward value of each cue stream with an update relationship is continuously updated. The expected reward value of the better-performing cue stream gradually increases, while the expected reward value of the worse-performing cue stream gradually decreases, thereby achieving adaptive learning and optimization of cue streams with update relationships.

9. The method for automatically constructing task-oriented intelligent agents based on natural language dialogue according to claim 1, characterized in that, In step S4, the second intelligent agent definition is a target intelligent agent generated by the large language model based on the natural language dialogue process. It is a solidified prompt stream, including policies, constraints, and preferences, and has the characteristics of being reusable and callable.

10. An automatic construction system for task-oriented intelligent agents based on natural language dialogue, characterized in that, This includes a prompt stream construction module, a prompt stream evaluation module, a prompt stream selection module, a prompt stream combination module, and a prompt stream freezing module; The prompting stream construction module is used to perform semantic analysis and feature extraction on each round of natural language dialogue between the user and the first intelligent agent to obtain structured semantic signals, and then construct a prompting stream based on the structured semantic signals. The prompting flow evaluation module is used to execute a predetermined task using the prompting flow constructed in each round of natural language dialogue. It quantitatively evaluates the execution effect of the current prompting flow based on three aspects: explicit feedback reward, implicit behavioral reward, and automatic evaluation reward, and calculates the comprehensive benefit value R of the current prompting flow. t ; The prompt flow selection module is used to select a set of prompt flows with an update relationship as a candidate prompt flow set; first, it obtains the initial expected benefit value of each prompt flow in the same candidate prompt flow set; then, it treats the single prompt flow constructed in this round of natural language dialogue as an optional action, based on its comprehensive benefit value R. t The expected payoff value of the suggestion stream is updated using a multi-armed slot machine strategy, while the expected payoff values ​​of the remaining suggestion streams in the candidate suggestion stream set remain unchanged. The prompt flow combination module is used to retain only the prompt flow with the highest expected return value for each candidate prompt flow set; prompt flows that do not constitute a candidate prompt flow set are retained as independent prompt flows; and all retained prompt flows are sorted from high to low according to their expected return values ​​to form a combined prompt flow execution sequence. The prompt stream freezing module is used to freeze the current combined prompt stream execution sequence when the user clearly confirms that the effect of the current combined prompt stream execution sequence can be reused for a long time, and to generate a second agent definition based on the combined prompt stream execution sequence.

Images