Method for performing agent reasoning based on skill library and computing equipment

Abstract

A method and computing device for agent reasoning based on a skill library, the skill library comprising a plurality of planning skills and a plurality of functional skills, each planning skill comprising a plurality of subtasks corresponding to a class of tasks, each functional skill corresponding to a class of subtasks, the method comprises the steps that for a first task to be reasoned, a first planning skill is retrieved from the skill library, and the first planning skill comprises a plurality of first sub-tasks; based on the plurality of first subtasks, retrieving from the skill library to obtain a plurality of first functional skills; reasoning based on the first task and the plurality of first functional skills through an intelligent agent to obtain a first track.

Description

Technical Field

[0001] The embodiments in this specification belong to the field of large model technology, and in particular relate to a method and computing device for agent reasoning based on a skill base. Background Technology

[0002] Large Language Models (LLMs), also known simply as large models, are natural language processing models based on deep learning techniques. Their parameter count typically ranges from billions to hundreds of billions or even higher, possessing powerful language understanding and generation capabilities. LLMs can employ the Transformer architecture or its variants (such as GPT and BERT), which utilizes an attention mechanism to globally model sequential data, efficiently handling long-distance dependencies and thus performing exceptionally well in natural language tasks. LLMs learn the statistical features and semantic relationships of language through pre-training on large-scale corpora, enabling them to generalize. The core capabilities of LLMs include, but are not limited to: understanding contextual semantics, generating coherent and grammatically correct text, performing logical reasoning, and handling multi-task scenarios. Their usage typically includes two modes: direct inference and fine-tuning. In direct inference mode, users design prompts to guide the LLM in generating specific outputs. Prompts can be textual descriptions of the task or instructions used to stimulate the LLM's semantic understanding and generation capabilities. In fine-tuning mode, large language models are further trained on small-scale datasets within a specific domain to optimize their performance on specific tasks. The powerful generalization capabilities and flexibility of large language models make them an important tool in the field of artificial intelligence, providing efficient and accurate solutions for automated text generation and understanding.

[0003] In some embodiments, large language models can also understand and generate data from other modalities (such as visual and audio data). In this case, large language models can also be called multimodal large language models (MLLMs). MLLMs provide a richer and more natural interactive experience by integrating multiple types of input and output, such as text, images, and sound. The core advantage of MLLMs lies in their ability to process and understand information from different modalities and fuse this information to complete complex tasks. For example, MLLMs can analyze an image and generate descriptive text, or generate a corresponding image based on a text description. This cross-modal understanding and generation capability makes MLLMs widely applicable across multiple fields.

[0004] It should be noted that the key technologies of large language models can be found in the paper "A Survey of Large Language Models" (paper number: arXiv:2303.18223v16, published on March 11, 2025). This specification will not elaborate on them here.

[0005] An agent, a concept in artificial intelligence and computer science, generally refers to an autonomous system or entity capable of perceiving its environment, making decisions, and taking actions to achieve a certain goal. Agents can be software programs, robots, or other intelligent systems. They accomplish specific tasks or goals by perceiving and interacting with their environment. Agents can acquire environmental information through tools such as sensors, cameras, and data interfaces to understand the current environmental situation; make decisions through algorithms or rule systems to determine what actions to take in a specific environment to achieve the goal; and execute corresponding operations through actuators or output interfaces. For example, an agent can use a Large Language Model (LLM) as a policy model to output actions based on observations / states.

[0006] Agents can invoke external tools (such as bank APIs) based on the Model Context Protocol (MCP). MCP provides a unified, machine-readable description for all external tools, essentially providing an interaction specification between agents in LLM applications and external services. The MCP architecture includes an MCP host, an MCP client, and an MCP server. The MCP host, typically referring to the AI ​​application, is the initiator of the interaction. The MCP client, located within the MCP host, is used to discover services provided by the MCP server and to transmit information between the LLM and the MCP server. The MCP protocol includes authentication and access authorization capabilities based on Open Authorization (OAuth). OAuth is an open security protocol that provides an authorization standard for applications to access user resources. Its key feature is the granting of resource access permissions through a token mechanism, avoiding the use of sensitive information such as account passwords.

[0007] For example, agents can be modeled as policies. The agent can obtain a trajectory through rollout for a specific task q. :

[0008] The agent can obtain the trajectory through reasoning over t time steps. In other words, a trajectory refers to the sequence of observations and actions generated by an agent during reasoning for a task. When the agent performs reasoning at the i-th time step, it bases its reasoning on the context... Choose an action ,in This refers to a set of executable actions, specifically a set of tools that an agent can invoke. As mentioned above, an agent can discover external tools provided by an MCP server based on an MCP client, thereby generating a tool set. For example, in a scenario where an agent such as "Financial Butler" interfaces with multiple banks, each bank's server can deploy an MCP server to provide the agent's MCP client with information related to tools (such as API interfaces).

[0009] Recent advancements in Large Language Model (LLM)-based agents have made significant progress in combining tools for long-term decision-making, enabling behaviors such as API calls, web navigation, scientific discovery, and interactive assistants. For example, an agent acting as a "financial steward" can handle a large number of business requests from different banks. These services have low-cost requirements, and each bank's operational costs are highly sensitive; therefore, it's impossible to train or fine-tune a dedicated model for each specific business scenario. Furthermore, different banks' services differ in detail; for example, the processes and rules for account inquiries, fund transfers, wealth management product recommendations, and loan application approvals vary slightly between banks. Therefore, considering these business characteristics, a single agent is needed to provide inference services for business requests from different banks simultaneously. In this scenario, due to the differences in business details between banks, it's difficult for the agent to apply successful experiences from bank A to tasks from bank B.

[0010] For example, in one related scheme, the agent can first infer multiple historical success trajectories based on a stronger model (hereinafter referred to as the strong model) through offline reasoning, and store these historical success trajectories as success experience. Then, for cost considerations, when providing services online through the agent, it can perform reasoning based on a weaker model (hereinafter referred to as the weak model). The weak model has fewer model parameters than the strong model, thus requiring less computational and storage resources. When executing a new task online, the agent retrieves similar trajectories from the historical success trajectories and performs reasoning based on these similar trajectories. However, the agent's historical success trajectories contain a lot of noise, such as erroneous actions, irrelevant information, and spurious relevance. The weak model in this agent cannot identify contradictions or irrationalities in the retrieval results. The weak model may mistake chance, erroneous operation, or environmental coincidence for key success, leading to incorrect reasoning results. For example, for the aforementioned "financial steward" agent, due to the differences in the details of different banks' business operations, the weak model may lead to incorrect reasoning results when processing a task from bank B based on bank A's historical success trajectories. Summary of the Invention

[0011] The purpose of this invention is to provide a method for agent reasoning based on a skill base, so as to enhance the use of reasoning experience by weak models.

[0012] The first aspect of this specification provides a method for agent reasoning based on a skill base. The skill base includes multiple planning skills and multiple functional skills. Each planning skill includes multiple sub-tasks corresponding to a type of task, and each functional skill corresponds to a type of sub-task and includes skill description information and tool invocation information corresponding to multiple tools. The method includes: for a first task to be reasoned about, retrieving a first planning skill from the skill base, the first planning skill including multiple first sub-tasks; retrieving multiple first functional skills from the skill base based on the multiple first sub-tasks; and obtaining a first trajectory by having an agent reason based on the first task and the multiple first functional skills.

[0013] A second aspect of this specification provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the method described in the first aspect.

[0014] A third aspect of this specification provides a computing device, including a memory and a processor, wherein the memory stores executable code, and the processor executes the executable code to implement the method described in the first aspect.

[0015] A fourth aspect of this specification provides a computer program product including a computer program / instructions that, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] The reasoning method described in the embodiments of this specification first retrieves planning skills from the skill base for the task to be reasoned about, and then retrieves functional skills from the skill base based on each subtask of the planning skills. The agent can then directly perform model reasoning based on these functional skills. Compared to situations where the agent reasons based on historical trajectories, the agent does not require strong reasoning capabilities; it only needs to understand the documentation of the functional skills. This allows the skill base described in the embodiments of this specification to be easily ported and applied to scenarios with weak models. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments in this specification, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a schematic diagram illustrating the process of creating a skill library in the embodiments of this specification;

[0019] Figure 2 This is a schematic diagram of the planning skills, functional skills, and atomic skills extracted from the embodiments of this specification;

[0020] Figure 3 This is a schematic diagram illustrating the skill refinement process in the embodiments of this specification;

[0021] Figure 4 This is a flowchart of agent reasoning based on a skill base in one embodiment of this specification;

[0022] Figure 5 This is a schematic diagram illustrating the process of agent reasoning based on a skill base in another embodiment of this specification;

[0023] Figure 6 This is a schematic diagram illustrating the iterative update process of the skill library in the embodiments of this specification. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0025] The embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0026] Figure 1 This is a schematic diagram illustrating the process of creating a skill library in the embodiments of this specification.

[0027] like Figure 1 As shown, firstly in step S103, the trajectory output by the agent for task q is... Extract skills from multiple levels.

[0028] The agent can perform multiple inferences for task q, obtaining multiple trajectories, and execute based on each successful trajectory among these multiple trajectories. Figure 1 The method shown is used to update the skill base. The successful trajectory is the trajectory of successfully executing a task. In this way, due to the stochastic nature of LLM inference, differentiated solution paths can be generated through multiple inferences, allowing the skill base to include multiple effective strategies (such as multiple implementation schemes of functional skills), thereby improving the agent's adaptability to new tasks. The trajectories mentioned below will be referred to as such. Let's take an example to describe it.

[0029] In one embodiment of this specification, skills at multiple levels may include planning skills and functional skills. Each planning skill includes multiple sub-tasks corresponding to a type of task. These multiple sub-tasks may correspond to multiple sub-tasks obtained by decomposing the type of task.

[0030] Each functional skill corresponds to a type of subtask, and each functional skill indicates the skills required to complete a specific subtask. Specifically, a functional skill may include skill description information and tool invocation information corresponding to multiple tools. This tool invocation information includes, for example, the input parameters and outputs of each tool when invoked.

[0031] In another embodiment of this specification, skills at multiple levels may include planning skills, functional skills, and atomic skills. Atomic skills are related to toolsets. The individual tool correspondence includes the tool's extended semantic specification and tool invocation information. The extended semantic specification may include, for example, enhanced descriptions, constraints, or usage examples to finely define the effective operation of the tool.

[0032] For a successful trajectory First, the trajectory can be analyzed. Compression is performed to reduce noise in the extracted skills during subsequent skill extraction and improve skill reusability. During compression, unnecessary state transitions, such as exploration, backtracking, and trial-and-error behaviors, can be filtered out. While these behaviors may occasionally appear in the final solution, they impair skill reusability and increase noise in the extracted skills.

[0033] Furthermore, for excessively long or redundant environmental feedback in the trajectory, summarization techniques can be used to generate concise state descriptions, thereby improving the stability and fidelity of the extracted planning skills. For example, the agent can output a summary of the environmental feedback based on the following prompts:

[0034] "You are an AI assistant that specializes in analyzing the trajectory of intelligent agents."

[0035] Your task is to summarize a single interaction: based on the environmental feedback of the current step, extract and summarize the key information in no more than 50 words.

[0036] Input instructions:

[0037] 1. Reasoning and Action of AI Assistants

[0038] 2. Environmental feedback after the action

[0039] Abstract Guidelines:

[0040] 1. Summarize the content conveyed by the environmental feedback based on the AI ​​assistant's intent.

[0041] 2. Retain as much detail as possible that is closely related to the intent; compress other redundant information.

[0042] 3. Only summarize the factual content of the environmental feedback; fabrication is not permitted.

[0043] 4. The tone of the summary should be consistent with the environmental feedback.

[0044] Output format:

[0045] <feedback>

[0046] Summary of your environmental feedback

[0047] < / feedback> "

[0048] The trajectory extraction and planning skills can then be obtained based on the above processing. Alternatively, the trajectory and corresponding task extraction and planning skills can be obtained based on the above processing.

[0049] Then, functional skills can be extracted based on the trajectory and the extracted planning skills. Specifically, functional skills corresponding to each subtask in the planning skills can be extracted. Functional skills can include three key fields: name (skill name), document (document), content (content), and tools (tool list).

[0050] Then it can be based on the trajectory and toolset Extract atomic skills. Atomic skills enhance the interface description of the original tool by incorporating reusable, execution-oriented usage patterns. When higher-level functional skills are missing or incomplete, atomic skills can serve as a supplement to lower-level ones.

[0051] In one embodiment of this specification, an agent can be used based on trajectory Extract planning skills, functional skills, and atomic skills.

[0052] Specifically, this can be achieved through intelligent agents based on the following prompts and trajectories. Extract planning skills from task q:

[0053] "You are a planning expert."

[0054] Your task is to analyze the agent's API interaction history and the user's tasks, and distill this into a concise, reusable plan. This plan should serve as a reference for handling similar tasks in the future, to improve efficiency and effectiveness.

[0055] Objectives

[0056] Understand Capabilities

[0057] • Analyze the recorded API calls to identify the actual functional capabilities exhibited.

[0058] Abstract into a Plan

[0059] • For each feasible task supported by these capabilities, generate a concise, reusable step-by-step plan that can be used to handle similar tasks.

[0060] Planning Creation Rules

[0061] Focus Intent

[0062] • Do not repeat the technical details or terminology of each API call verbatim.

[0063] • Each operational paragraph should describe the sub-goal behind it, i.e., "why this step should be performed".

[0064] Remove non-essential steps.

[0065] • Eliminate the process of exploring and debugging capabilities, as well as the steps to overcome failed attempts.

[0066] Ensure reusability.

[0067] • The plan must be clear and accurate enough so that other models can be directly reused.

[0068] Keep it simple (Conciseness)

[0069] • Combine multiple steps with the same goal in the interaction history into a sub-step in the plan.

[0070] • Each sub-step is described in concise language and lists the key APIs (one or more) involved in that step.

[0071] • No critical, potentially necessary API parameters or fields may be omitted.

[0072] Output format (OUTPUT FORMAT)

[0073] For each task, output only one plan, strictly adhering to the following format:

[0074] # Step 1: A natural, specific, and concise subtask goal; involving one or more key APIs.

[0075] # Step 2: ... ...

[0076] Good Examples

[0077] {examples}

[0078] Final confirmation checklist (before finalizing)

[0079] Reusability — Ensure no critical steps are missing and that the steps are in the correct order.

[0080] Simplicity — Ensure there are no redundant or unnecessary steps.

[0081] Centered on the agent—ensure the plan reads like operational instructions that other models can reliably execute.

[0082] As can be seen from the above prompt, the prompt instructs the agent to base its actions on the trajectory. Task q extracts multiple subtasks corresponding to a class of tasks, so that the extracted planning skills can be reused in other similar tasks. In one embodiment, the agent can also be instructed in the prompt to plan based on the trajectory. The task q and skill library extract planning skills, so as to extract only planning skills that are not similar to the planning skills already existing in the skill library.

[0083] The agent can extract functional skills based on the following prompts:

[0084] "An intelligent agent system is equipped with a skill library and has attempted to solve a certain task multiple times, eventually achieving a successful solution. Please review the task-solving process and extract generalizable skills from it."

[0085] 1. Input Instructions

[0086] User Task

[0087] Trajectory: A complete record of successful interactions between an agent and its environment as it attempts to complete a user task.

[0088] Skills library: The collection of all currently usable skills.

[0089] Specific-step: Given a specific step, extract a reusable skill for that step only.

[0090] 2. Skill Definition Rules

[0091] Skills is a dictionary with four keys: name, document, content, and tools.

[0092] name: The name of the skill.

[0093] document: Describe the function of the skill, key parameters, final output, and any important notes.

[0094] content: The specific implementation logic of the skill.

[0095] tools: Key tools used in the skill (in list form).

[0096] Skills must be abstract, modular, and reusable. Specific requirements are as follows:

[0097] Skill names should be generic in a single application scenario (e.g., "Get songs by genre on spotify" instead of "Get popular songs").

[0098] Parameters must be used instead of hard-coded values ​​(e.g., you should not hardcode email addresses like "jay@gmail.com").

[0099] The skill itself must be self-contained.

[0100] You must explicitly declare the data type of key parameters and the final output using type hints. For example:

[0101] Parameters: param: str; Outputs: output: list[dict]

[0102] It must include a detailed description of the skill, including explanations of the inputs and outputs.

[0103] The extracted skills must not be similar to skills already in the skill library.

[0104] A skill must involve multiple processing steps. Simply calling an API and returning a result directly (without additional logic) does not constitute a valid skill.

[0105] Calling other skills from the skill library or any previously defined skills is prohibited.

[0106] It is understood that the above prompt assumes that the skill library already contains skills, so that the agent can be instructed to extract the current functional skills based on the skills already in the skill library.

[0107] As can be seen from the above prompt, this prompt instructs the agent to improve the modularity and reusability (or versatility) of functional skills by limiting skill names and parameters. With this setting, each functional skill can be reused in tasks of different scenarios.

[0108] Atomic skills can be extracted by an agent based on the following prompts:

[0109] "An intelligent agent system is equipped with a skill library and has attempted to solve a certain task multiple times, eventually achieving a successful solution. Please review the task-solving process and extract generalizable skills from it."

[0110] 1. Input Instructions

[0111] User Task

[0112] Trajectory: A complete record of successful interactions between an agent and its environment as it attempts to complete a user task.

[0113] Skills library: The collection of all currently usable skills.

[0114] Specific-Tool: Given a specific tool, extract a reusable skill only for that tool.

[0115] 2. Skill Definition Rules

[0116] A skill is a dictionary with four keys: name, document, and content.

[0117] name: The name of the specific tool.

[0118] document: Describes the tool's functionality, key parameters, final output of the skill, and any important considerations.

[0119] content: Examples of how this tool is used, and (where applicable) examples of its use in combination with other tools.

[0120] The skills should revolve around a specific tool, describing its core functions, important considerations, and common usage examples.

[0121] Type hints must be used to explicitly declare the data types of key parameters and the final output. For example: Parameters: param: str; Outputs: output: dict

[0122] It must include a detailed description of the skill, including explanations of the inputs and outputs.

[0123] The extracted skills must not be similar to skills already in the skill library.

[0124] The parameters used in the content must be reusable, not hard-coded values ​​(e.g., email addresses "jay@gmail.com" should not be hardcoded).

[0125] The usage examples in the content may involve one or more tool calls.

[0126] The documentation must clearly and comprehensively record all relevant details of the use of this particular tool.

[0127] Calling other skills from the skill library or any previously defined skills is prohibited.

[0128] Figure 2 This is a schematic diagram of the planning skills, functional skills, and atomic skills extracted from the embodiments of this specification.

[0129] Suppose the task q above is: to transfer money through my bank account. The intelligent agent uses task q and its corresponding trajectory... The extracted planning skills can include the following sub-tasks:

[0130] Step 1: Log in to the **bank online banking platform;

[0131] Step 2: Locate the transfer function;

[0132] …

[0133] Step 5: Submit and confirm the transfer result.

[0134] Based on task q and trajectory, the intelligent agent... One or more functional skills corresponding to each subtask can be extracted. For example, such as Figure 2 As shown, the name of the functional skill corresponding to step 1 is "Log in to the online banking platform." The document for this functional skill includes a description of the skill, input parameters, output data, and precautions. The content of this functional skill includes the specific implementation logic of the skill. Figure 2 (Not shown in the image), specifically including tool invocation modes and parameter configurations (such as input parameters and output results). The tools section for this function / skill includes a list of tools used by this function / skill.

[0135] Based on task q and trajectory, the intelligent agent... Multiple atomic skills can be extracted. For example... Figure 2As shown, an atomic skill includes the skill's name, document, and content. The document describes the skill's function, key parameters, final output, and any important considerations. These considerations may include constraints and common failure modes observed during actual tool use. The content may include one or more examples of tool usage, examples of combining it with other tools, etc.

[0136] Back Figure 1 In step S103, the skills of the multiple trajectories extracted above can be refined.

[0137] Figure 3 This is a schematic diagram illustrating the skill refinement process in the embodiments of this specification.

[0138] After extracting multi-level skills from multiple trajectories corresponding to the aforementioned task q, a large number of redundant skills are usually obtained. These redundant skills, although different in form, essentially correspond to the same skills. Therefore, as... Figure 3 As shown on the left, redundant skills can first be fused. For example, for the multiple functional skills extracted in the above steps, each functional skill can first be converted into an embedded representation (or embedding vector) of text. Then, based on the embedded representations of each functional skill, multiple functional skills can be clustered to obtain multiple clusters. Multiple functional skills in a cluster are equivalent to providing multiple complementary update directions for the same functional skill. Afterwards, the agent can output the fused functional skill corresponding to the cluster as a single functional skill based on the multiple functional skills in the cluster. After outputting the fused functional skill, the agent can judge whether the fused functional skill is too complex based on prompts. If it is too complex, it can be further decomposed into several sub-functional skills.

[0139] For example, an agent can fuse multiple functional skills within a cluster based on prompts such as:

[0140] "You are a code expert. Your task is to analyze a set of skills, merge semantically similar skills, and break down overly complex skills into smaller atomic skills while preserving their original behavior and intent."

[0141] Input instructions

[0142] Users will provide a list of skills.

[0143] Skill definition rules

[0144] Skills is a dictionary with four keys: name, document, content, and tools.

[0145] name: The name of the skill.

[0146] document: Describe the function of the skill, key parameters, final output, and any important notes.

[0147] content: The specific implementation logic of the skill.

[0148] tools: Key tools used in the skill (in list form).

[0149] Skills must be abstract, modular, and reusable. Specific requirements are as follows:

[0150] Skill names must be generic in a single application scenario (e.g., use {good example} instead of {bad example}).

[0151] Parameters must be used instead of hard-coded values ​​(e.g., email addresses should not be hardcoded).

[0152] The skill itself must be self-contained.

[0153] You must explicitly declare the data type of key parameters and the final output using type hints. For example: Parameters: param: str; Outputs: output: list[dict]

[0154] It must include a detailed description of the skill, including explanations of the inputs and outputs.

[0155] Skills must not be duplicates or highly similar to skills already in the skill library.

[0156] A skill must involve multiple processing steps. Simply calling an API and returning a result directly (without additional logic) does not constitute a valid skill.

[0157] Calling other skills (whether from the skill library or previously defined skills) is prohibited.

[0158] No Python packages may be imported.

[0159] Avoid functional style; there's no need to use `return`.

[0160] Examples of excellent skills:

[0161] json

[0162] {

[0163] "name": {name},

[0164] "document": {document},

[0165] "content": {content},

[0166] "tools": {tools}

[0167] }

[0168] Key points of analysis

[0169] Pay attention to skills with similar names and similar functions.

[0170] Carefully analyze the differences in the specific implementation (content) of similar skills.

[0171] Merge Guidelines

[0172] Generality: Merge skills with similar names and functions. The merged skills should use a common name, and their documentation and implementation logic should cover all reasonable variations and edge cases.

[0173] Atomicity: If there is a containment relationship between skills (e.g., one skill's functionality completely includes or is built upon another), they should not be merged, but should remain independent in accordance with the principle of atomicity.

[0174] Merge constraints:

[0175] Any merged skill must comply with the skill definition rules, especially ensuring atomicity and reusability, and avoiding strong binding to specific tasks or scenarios.

[0176] Decompose Guidelines

[0177] Atomicity: A skill is broken down into smaller sub-skills only when its function is too complex (e.g., its function already covers the function of other existing skills).

[0178] Generality: The decomposed sub-skills must comply with the skill definition rules and maintain reusability—they must not be coupled to any specific task or scenario.

[0179] Output format

[0180] Output a list containing one or more skills obtained by merging and / or disassembling the input skill list, in the following format:

[0181] [ "skill 1", ... ]”

[0182] Still referencing Figure 3After skill fusion, you can also filter the current multiple skills. (Reference) Figure 3 On the right, multiple skills can first be filtered for generality to remove skills that lack portability or composability, including those that rely on external code (such as external Python packages), contain overly specific functional definitions, contain hard-coded values, or are overly encapsulated.

[0183] Then it can be based on the toolset Tool call filtering is performed by comparing each functional skill or atomic skill with the tool interface specification provided by the environment to exclude skills that reference non-existent tools, invalid parameters, or parameter structures that are incompatible with the interface specification.

[0184] For example, an agent can filter tool calls for multiple functional skills or atomic skills based on the following prompts:

[0185] "You are a tool invocation expert. Please verify that the provided tool invocation is correct according to the tool specification."

[0186] Input content:

[0187] Tool call content: May contain one or more tool calls.

[0188] Tool specifications: including tool description, parameters, return value structure and other usage instructions.

[0189] Judgment criteria:

[0190] Parameter validation: Check whether the called parameters conform to the specifications (e.g., whether required parameters are missing, whether unsupported or non-existent parameters are included, whether the type or format is incorrect, or whether the value is invalid).

[0191] Call Dependency: For multiple tool calls, verify that their order does not violate logical dependencies; if there are no dependencies between calls, ignore this check.

[0192] Consistency between comments and functionality: Ensure that the logic described in any comments is consistent with the functionality designed for the tool.

[0193] Output format: Please first explain your reasoning process, and conclude with "correct" or "fail" in a label.

[0194] Back Figure 1 In step S105, the skill library can be updated based on the refined planning skills, functional skills, and atomic skills.

[0195] When the skill library is empty, the refined planning skills, functional skills, and atomic skills can be directly stored in the skill library.

[0196] If the skill base already includes skills, for example, multiple refined skills can be compared with existing skills in the skill base. If the similarity between any of the refined skills and a skill in the skill base is less than a preset threshold, that skill can be added to the skill base. Skills in the skill base can be modified based on the refined skills to improve their effectiveness. Alternatively, if the similarity between the refined skills and a skill in the skill base is greater than a preset threshold, the skill base can remain unchanged.

[0197] For example, an agent can update its skill set based on prompts such as:

[0198] 1. Update existing skills

[0199] Your goal is to ensure the system retains actionable skills that will contribute to correct future behavior. You have three action options: [modify, add, keep].

[0200] Modify: Revise existing skills to improve their effectiveness (e.g., improve documentation). Only modify the content when necessary, and ensure that the modified skills remain widely reusable / universal.

[0201] add: New skills are introduced only when existing skills cannot support a critical step, in order to improve future performance.

[0202] Keep: When there are no obvious problems with existing skills, keep them unchanged.

[0203] Common operations include:

[0204] Add a new skill

[0205] Update skill usage instructions / documents

[0206] Revise the variable / parameter definitions of the skill to make it more generalizable.

[0207] If a skill is too complex, refactor it into multiple more modular skills (involving modification and addition).

[0208] Keep skills unchanged

[0209] 2. Requirements for each modified or added skill.

[0210] Avoid duplication: If a skill library is provided, new skills that are similar to existing skills should not be added—choose keep or modify.

[0211] Exclude non-solution behaviors: No capability exploration, debugging activities, or any failed / error steps are allowed.

[0212] Ensure domain specificity: Skills must reference domain-specific libraries / APIs, such as {api}.

[0213] Avoid over-encapsulation: Ensure that the implementation is not a simple wrapper around another skill (i.e., it cannot just call a low-level skill without any substantial additional logic).

[0214] Extraction based on specific steps: When extracting skills, focus only on the specific step specified in the trajectory.

[0215] 3. Examples of High-Quality Skills

[0216] {example}

[0217] 4. Output Format

[0218] Finally, please return the results in JSON format.

[0219] Note: The skills you update do not necessarily need to cover all three options. You can use only one update type, or you can choose not to change any skills at all.

[0220] 5. Final Confirmation Checklist

[0221] Reusability — Ensure no critical steps are missing, each skill is modularly designed, and all parameters are abstract rather than concrete values.

[0222] Standardization — Ensure that each skill meets the standards defined above.

[0223] Agent-centric approach—Add helpful tips and explanations to each skill to aid in the correct use by other models.

[0224] Focus on specific steps—does the extracted skills include content that does not belong to that step?

[0225] In passing Figure 1 After creating the skill library using the process shown, you can access it through methods such as... Figure 4 The flowchart shown illustrates agent reasoning based on a skill library.

[0226] like Figure 4 As shown, in step S401, for the task q′ to be reasoned, a planning skill is retrieved from the skill base, which includes multiple sub-tasks.

[0227] One or more planning skills can be retrieved from the skill base by calculating the similarity between task q′ and each planning skill in the skill base. For example, the similarity between task q′ and each planning skill in the skill base can be calculated by calculating the similarity between the embedding representation of task q′ and the embedding representations of each planning skill. A retrieved planning skill includes multiple subtasks as follows: {step1, step2, ..., step...} p}

[0228] In step S403, multiple functional skills are retrieved from the skill library based on multiple sub-tasks.

[0229] Each subtask in the planning skill can be viewed as a retrieval query to retrieve the corresponding functional skill. For example, for the i-th subtask step... i It can be based on subtask step i The skill set is retrieved by comparing the similarity with each functional skill. Then, duplicate results can be deduplicated for all subtasks:

[0230] In one implementation, to maintain contextual brevity and high task relevance, an agent can self-filter the deduplicated candidate skills to retain those suitable for the current task, resulting in a final set of functional skills for solving task q′. .

[0231] In step S405, the agent performs inference based on task q′ and multiple functional skills.

[0232] For example, task q′ and set Multiple functional skills are concatenated into a prompt, which is then input into the agent, allowing the agent to use a set of skills. Using the functional skills in the text as conditions, reasoning is performed to obtain the trajectory. : .

[0233] exist Figure 4 In the inference method shown, planning skills are first retrieved from the skill base for the task to be inferred. Then, functional skills are retrieved from the skill base based on each subtask of the planning skills. The agent can then directly perform model inference based on the functional skills. Compared to the case where the agent infers based on historical trajectories, the agent does not need strong reasoning ability; it only needs to understand the documentation of functional skills. This makes the skill base in the embodiments of this specification easily applicable to scenarios with weak models.

[0234] In another implementation, in step S403, multiple functional skills and atomic skills can be retrieved from the skill library based on multiple sub-tasks, and in step S405, the agent can perform reasoning based on task q′, multiple functional skills and multiple atomic skills.

[0235] Figure 5 This is a schematic diagram illustrating the process of agent reasoning based on a skill base in another embodiment of this specification.

[0236] When task q′ is novel and complex, directly recalling planning skills based solely on task similarity may result in retrieved experience not matching the actual task. To improve the relevance of subsequent skill retrieval, such as... Figure 5 As shown, after retrieving multiple planning skills based on task q′, an agent can rewrite the plan based on these multiple planning skills to obtain a rewritten planning skill adapted to task q′. This rewritten planning skill, for example, includes multiple subtasks { , ,…, Then it can be used with... Figure 4 The method shown is similar, based on multiple subtasks { , ,…, The skill set is retrieved, and then deduplication and agent self-selection are performed on the retrieved skill set to obtain the final skill set. This skill set It may include multiple functional skills, or it may include multiple functional skills and multiple atomic skills. Then, the agent can determine the skill set based on the task q′ and the skill set. To reason.

[0237] In this way, the planning skills retrieved by the agent are rewritten into multiple subtasks that are more closely matched to task q′. , ,…, }, thus enabling multiple subtasks based on this rewrite { , ,…, When retrieving functional skills, results can be obtained that are more relevant to task q′. This allows the agent to be based on the task q′ and the skill set. Trajectories obtained through reasoning yield better reasoning results. For example, in the scenario of a "financial steward" agent, when task q′ is a task associated with bank B, if the retrieved planning skill is extracted based on the historical successful trajectory of bank A, the agent can rewrite multiple subtasks in this planning skill based on the relevant processes of bank B. This rewritten subtasks match the processing flow of bank B, and the functional skills retrieved based on these rewritten subtasks are more helpful for the agent to make correct reasoning for task q′.

[0238] In the embodiments described in this specification, the trajectory is obtained by an intelligent agent inferring from task q′ based on a skill base. After that, you can pass through again. Figure 1 The method shown is based on trajectory Update the skill library, thereby iterating and updating the skill library multiple times.

[0239] Figure 6 This is a schematic diagram illustrating the iterative update process of the skill library in the embodiments of this specification.

[0240] For this iterative update process, a training sample set can be prepared in advance. and test sample set Among them, the training sample set It includes multiple tasks and a test sample set. It includes multiple tasks and the reference trajectory (i.e., the true trajectory) for each task.

[0241] Firstly, it can be based on the training sample set. Multiple training samples are used to iterate through the skill base in one round. This round of iteration includes multiple updates to the skill base, such as in the k-th update. Figure 6 As shown in the "Reasoning and Skill Extraction" module, it can be obtained from the training sample set. Mid-sampling task, through Figure 4 or Figure 5 The method shown ( Figure 6 The middle is shown as Figure 5 The method described in the text retrieves functional skills and atomic skills based on a skill base. The agent then uses these functional and atomic skills to infer a trajectory. Afterwards, it can proceed according to... Figure 1 The method for updating the skill library shown extracts planning skills, functional skills, and atomic skills from the trajectory by the agent. Figure 3 The method shown refines the extracted skills, and then the skill library can be updated based on the refined skills.

[0242] Then it can be based on the test sample set Test the skillset. For example, for a test sample set. Several test samples in the sample can be obtained through Figure 6 The reasoning and skill extraction processes in the algorithm involve reasoning on each test task based on a skill base to obtain the predicted trajectory for each test sample. Then, the inference loss is calculated based on the difference between the test trajectory and the reference trajectory for each test sample. The iteration can be terminated when the inference loss is less than a preset threshold, or when the inference loss no longer decreases compared to the previous iteration.

[0243] It is possible Figure 6 The method described involves periodically acquiring a training sample set to update the skill base. This allows for continuous updates to the skill base to address emerging task vulnerabilities, ensuring that the skill base consistently improves the agent's inference performance. During iterative updates, the agent can perform offline inference based on a strong model. After the skill base update is complete, it can be used in an online inference agent that performs inference based on a weak model.

[0244] While the skill set can be iteratively updated through the process described above, relying solely on limited training data is far from sufficient in scenarios with vast tool spaces and complex environments. Therefore, one embodiment of this specification employs… Figure 6 The "Experience-Guided Exploration" strategy shown is used to break through the training sample set. The scope of behavior is expanded to encourage agents to interact with the environment and try a wider range of tools.

[0245] During reasoning, the agent can record its understanding of the tool set. Historical usage data for the tools, such as the failure rate and the number of times each tool was called. (Reference) Figure 6 The "Task Expansion" module can first guide task exploration based on tool usage experience. Specifically, it can prioritize tools with fewer than a preset threshold of usage frequency and higher than a preset threshold of failure rate within the exploration trajectory, based on existing historical usage data of existing tools. Then, tasks and reference trajectories can be generated based on the exploration trajectory. For example, an agent can generate new tasks and reference trajectories based on the exploration trajectory. These generated tasks and reference trajectories can be used as training or testing samples.

[0246] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must also be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0247] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0248] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or physical entities, or by products with certain functions. A typical implementation device is a server system. Of course, this application does not exclude the possibility that, with the future development of computer technology, the computer implementing the functions of the above embodiments can be, for example, a personal computer, a laptop computer, an in-vehicle human-machine interaction device, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0249] While one or more embodiments of this specification provide the operational steps of the methods described in the embodiments or flowcharts, more or fewer operational steps may be included based on conventional or non-inventive means. The order of steps listed in the embodiments is merely one possible order of execution among many steps and does not represent the only possible order. In actual device or end product execution, the methods shown in the embodiments or drawings may be executed sequentially or in parallel (e.g., in a parallel processor or multi-threaded processing environment, or even a distributed data processing environment). The terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, product, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, product, or apparatus. Without further limitations, the presence of other identical or equivalent elements in the process, method, product, or apparatus that includes said elements is not excluded. For example, the use of terms such as "first," "second," etc., is to denote names and does not indicate any particular order.

[0250] For ease of description, the above devices are described in terms of function, divided into various modules. Of course, when implementing one or more of these specifications, the functions of each module can be implemented in one or more software and / or hardware components, or a module that performs the same function can be implemented by a combination of multiple sub-modules or sub-units. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, indirect coupling or communication connection between devices or units, and may be electrical, mechanical, or other forms.

[0251] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0252] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0253] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0254] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0255] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0256] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage, graphene storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0257] Those skilled in the art will understand that one or more embodiments of this specification can be provided as a method, system, or computer program product. Therefore, one or more embodiments of this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, one or more embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0258] One or more embodiments of this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a particular task or implement a particular abstract data type. One or more embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0259] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, system embodiments are basically similar to method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments. In the description of this specification, the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described can be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0260] The above description is merely an embodiment of one or more embodiments of this specification and is not intended to limit the scope of these embodiments. Various modifications and variations can be made to these embodiments by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims.

Claims

1. A method for agent reasoning based on a skill base, wherein the skill base includes multiple planning skills and multiple functional skills, each planning skill includes multiple sub-tasks corresponding to a type of task, each functional skill corresponds to a type of sub-task, and includes skill description information and tool invocation information corresponding to multiple tools, the method comprising: For the first task to be reasoned, a first planning skill is retrieved from the skill base, and the first planning skill includes multiple first sub-tasks; Based on the multiple first sub-tasks, multiple first functional skills are retrieved from the skill library; The first trajectory is obtained by the intelligent agent inferring based on the first task and the multiple first functional skills.

2. The method according to claim 1, wherein retrieving multiple first functional skills from the skill base based on the multiple first sub-tasks includes: Based on the first task, the plurality of first subtasks are rewritten to obtain a plurality of second subtasks; Based on the similarity between each of the second subtasks and each of the functional skills included in the skill library, the plurality of first functional skills are retrieved from the skill library.

3. The method according to claim 1, wherein the skill library further comprises a plurality of atomic skills, each atomic skill comprising skill description information and tool call information corresponding to a single tool, and the method further comprises: Based on the similarity between each second subtask and each of the atomic skills, multiple first atomic skills are retrieved from the skill library; The reasoning performed by the agent based on the first task and the multiple first functional skills includes: The agent performs inference based on the first task, the plurality of first functional skills, and the plurality of first atomic skills.

4. The method according to claim 3, further comprising: The agent extracts a second planning skill from the first trajectory, the second planning skill including multiple third sub-tasks; The intelligent agent extracts multiple second functional skills based on the multiple third sub-tasks; The skill library is updated based on the second planning skill and the plurality of second functional skills.

5. The method according to claim 4, wherein obtaining the first trajectory by reasoning through an intelligent agent based on the first task and the plurality of first functional skills includes: The intelligent agent performs multiple inferences based on the first task and the multiple first functional skills to obtain multiple first trajectories; The step of extracting the second planning skill from the first trajectory through the intelligent agent includes: The agent extracts multiple second planning skills from the multiple first trajectories respectively.

6. The method according to claim 4, wherein updating the skill library based on the second planning skill and the plurality of second functional skills comprises: The multiple second functional skills are clustered to obtain multiple clusters; The agent aggregates the second functional skills in each cluster to obtain multiple third functional skills; The skill library is updated based on the aforementioned multiple third-function skills.

7. The method according to claim 6, wherein updating the skill library based on the second planning skill and the plurality of second functional skills comprises: The multiple third-party functional skills are filtered based on any of the following: function definition, whether it contains hard-coded values, whether it depends on external code, encapsulation format, and tool call information; The skill library is updated based on multiple fourth-function skills obtained after filtering.

8. The method according to claim 7, wherein updating the skill library based on a plurality of filtered fourth functional skills comprises any one of the following: modifying functional skills in the skill library based on the plurality of fourth functional skills, adding new functional skills to the skill library based on the plurality of fourth functional skills, and keeping the functional skills in the skill library unchanged.

9. The method according to claim 1, further comprising: Based on the agent's historical call data for tools, a target tool is determined, wherein the failure rate of the target tool is higher than a preset threshold, or the number of calls to the target tool is lower than a preset threshold; Construct exploration trajectories based on target tools; The agent generates a second task based on the exploration trajectory, and the second task is used to update the skill library.

10. A computing device comprising a memory and a processor, wherein the memory stores executable code, and the processor, when executing the executable code, implements the method of any one of claims 1-9.

Images