A skill evolution method and system for autonomous search agents
By using a domain-based repair and two-stage filtering mechanism for externally searched skill documents, the problem of skill evolution in the model-frozen scenario of autonomous search agents was solved, achieving stable repair results and search optimization, and improving answer quality.
Patent Information
- Application Number
- CN202611433306.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-16
- Publication Date
- 2026-11-17
AI Technical Summary
Existing technologies rely heavily on model retraining in the skill evolution of autonomous search agents, making them unsuitable for model freeze scenarios. The mixed handling of error types leads to unstable repair results, and there is a lack of rule-based side effect control and unconstrained search costs.
By using a domain-based repair and two-stage screening mechanism based on externally searched skill documents, local revision measures are generated independently. Combined with pre-merge net gain verification and post-merge side effect testing, the stability and security of the repair are ensured.
It effectively addresses abnormal errors across domains, controls rule side effects, optimizes search decision logic, significantly reduces invalid retrieval and timeout issues, and improves answer quality without altering the underlying model.
Smart Images

Figure CN123115349A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of artificial intelligence and intelligent agent technology, specifically relating to a skill evolution method and system for autonomous search intelligent agents. Background Technology
[0002] Retrieval-enhanced question-answering systems are currently one of the mainstream architectures for large language models to handle knowledge-intensive tasks. These systems, by introducing external information sources to assist model generation, alleviate to some extent the limitations of the timeliness and coverage of the model's inherent knowledge. However, in complex multi-hop question-answering scenarios, the core challenge for these systems is not merely obtaining search results, but rather dynamically determining the search direction, evaluating the sufficiency of existing evidence, and terminating the search at appropriate times. These behavioral-level decision-making mechanisms are typically carried out by manually written prompts or external behavior configuration files, forming a replaceable control layer beyond the model parameters.
[0003] Retrieval-enhanced question answering systems are the mainstream architecture for large language models to handle knowledge-intensive tasks. They compensate for the inherent limitations of the model in terms of knowledge timeliness and coverage by accessing external information sources. In complex multi-hop question answering scenarios, the core challenges of the system lie in dynamically determining the retrieval direction, evaluating the sufficiency of evidence, and controlling the timing of search termination. These decision-making behaviors are generally controlled by manually written prompts or external behavior configuration files.
[0004] Currently, mainstream optimization solutions fall into two categories: one is to embed the search control mechanism into the model training target or fixed prompt words, with representative technologies including ReAct, IRCoT, Self-RAG, Adaptive-RAG, and CRAG. This type of solution requires simultaneous adjustment of model parameters and cannot adapt to application scenarios where the underlying model is frozen. The other type is to mine skill rules from the agent's running trajectory, such as SkillNet, Trace2Skill, and Evo Skill. However, this type of solution often mixes different types of abnormal trajectories, easily generates conflicting rules, and the repair effect is unstable.
[0005] In the prior art, CN121859943A discloses a method for extracting reusable experience and tools from historical task execution records and iteratively refreshing the internal experience base of an agent. This scheme filters successful and failed paths during execution and uses failure samples to guide the updating of the experience base and tool library. Its shortcomings are: the update objects are limited to the internal experience and tool storage area of the agent, and do not involve independently replaceable external behavior configuration files; at the same time, all failure records are mixed and processed, lacking a mechanism for distributed repair based on the cause of failure.
[0006] CN121902950A discloses a reinforcement learning scheme that enhances model policies by utilizing failure inference trajectories. This scheme constructs corrected samples by diagnosing the causes of failure and adjusts model policy parameters based on the corrected execution rewards. However, this scheme requires direct modification of model weights and is not suitable for application constraints where the underlying model is frozen or only external search skill documents can be replaced.
[0007] CN122173690A discloses a scheme for extracting historical retrieval strategies semantically similar to the current query from a local experience base, which stores strategies inferred from historical reasoning behavior. This scheme focuses on matching and reusing existing strategies, does not involve the ability to actively generate and filter new strategies from failure experiences, and does not establish a rule generation process based on error type.
[0008] CN122174861A discloses a scheme that breaks down an intelligent agent into standardized functional modules and dynamically reassembles these modules based on task feedback. This scheme is geared towards assembling the intelligent agent from scratch and continuously adjusting its structure. It does not design a dedicated external search skill evolution pipeline for the subdomain of autonomous search intelligent agents, nor does it introduce a risk control mechanism for rule side effects.
[0009] In summary, existing technologies have at least the following shortcomings: First, they are highly dependent on model retraining, and most solutions require parameter updates to the underlying model, making them unsuitable when the model is frozen; second, they handle error types in a mixed manner, merging and analyzing execution failures from different causes, leading to interference between repair rules and unstable repair results; third, they ignore rule side effects, as newly introduced behavioral rules may disrupt previously correctly handled issues, lacking systematic protection against rule overcorrection; and fourth, they are unconstrained by search costs, lacking monitoring of search counts, timeout termination, and other behavioral overheads during the repair process, making optimization direction and efficiency uncontrollable. Summary of the Invention
[0010] To address the shortcomings of the existing technologies, the present invention aims to provide a skill evolution method and system for autonomous search agents. This method uses external search skill documents as the sole optimization target, without altering the underlying model and retrieval components throughout the process, and achieves domain-specific repair of abnormal errors, prevention and control of rule side effects, and management of search behavior overhead.
[0011] This invention is achieved through the following technical solution: A skill evolution method for autonomous search agents includes the following steps: Step 1: Based on the initial search skills document, drive the autonomous search agent to perform retrieval-enhanced question-answering tasks on the target question set and generate an operation log; Step 2: Based on the evaluation of the operation log, divide the samples into normal samples and abnormal samples, and decompose the abnormal samples into different problem domains. Step 3: For each subset of abnormal samples corresponding to a problem domain, independently generate local revision measures that only apply to this problem domain, and apply the local revision measures to the initial search skill documents to generate candidate search skill documents for each problem domain. Step 4: For each candidate search skill document, a two-stage constraint screening mechanism consisting of pre-merging net gain verification and post-merging side effect testing is adopted. At the same time, the benefits of local repair and the risks of global regression are evaluated to complete the skill evolution and output the final search skill document.
[0012] The present invention also has the following technical features: Preferably, in step one, for each problem... The execution trajectory of a retrieval-enhanced question-answering system can be represented as follows: in, That's the standard answer. It is system output. and They represent the first Wheel movement and retrieval observation, It is the number of execution rounds. Record whether the execution ended successfully, whether the maximum number of rounds was triggered, and other statuses.
[0013] Preferably, the abnormal sample set in step two is represented as follows: in, Represents a set of abnormal samples. For sample index, For the first The execution trajectory of each problem This is the function for evaluating the answer; 0 indicates that the evaluation result is a failure.
[0014] Preferably, the problem domain in step two includes the problem parsing deviation domain, the search path misconception domain, the query construction defect domain, and the stop judgment timing misalignment domain; The formula for decomposing outlier samples into domains is expressed as follows: in, For the set of problem domains, To be included in the domain A subset of abnormal behavior records.
[0015] Preferably, each revision measure described in step three only modifies the part of the initial search skills document related to the problem domain, without changing other search control rules across domains.
[0016] Preferably, the pre-merging net gain verification process in step four includes: loading each candidate revision measure separately into the autonomous search agent system, rerunning it on the anomalous sample subset of its source problem domain, and statistically analyzing the repair ratio; This measure is eligible to proceed to the next stage only when the repair ratio reaches a preset entry threshold. The determination formula is as follows: in, For the set of samples that change from negative to positive in this domain, This is the threshold for the first-stage repair rate.
[0017] Preferably, the post-combination side effect testing process described in step four includes: All candidate revisions that passed the pre-merged net gain check were programmatically merged into the initial search skills document and fully validated on the complete sample set; the merging method is represented as follows: in, For the merged skills documents, This is an initial basic skills document. For programmed splicing operations, For the set of problem domain indexes that have passed the pre-merge verification, For the problem domain The revision measures.
[0018] The total number of repaired samples and the total number of degraded samples after merging are statistically analyzed. Only when the ratio of the number of repaired samples to the number of degraded samples reaches a preset safety threshold is the merged version officially accepted as the final evolved search skill document; the judgment formula is: in, This represents the total number of repaired samples. The total number of degraded samples, It is a very small positive number, used to prevent the denominator from being zero. This is the second-stage safety threshold.
[0019] The present invention also protects a system for performing the above method, including a baseline operation log module, an anomaly domain processing module, a candidate measure generation module, a two-stage screening module, and a skill deployment module; The baseline operation log module is used to drive the autonomous search agent to perform retrieval-enhanced question-answering tasks on the target question set based on the initial search skill document, and to collect and generate operation logs; The anomaly domain processing module is connected to the baseline operation log module. It is used to complete the effect evaluation based on the operation log, divide normal samples and abnormal samples, and decompose the abnormal samples into multiple independent problem domains. The candidate measure generation module is connected to the anomaly domain processing module. It is used to independently generate local revision measures that only apply to the corresponding problem domain for each subset of anomaly samples, and to adapt the local revision measures to the initial search skill document to generate candidate search skill documents for each problem domain. The two-stage screening module is connected to the candidate measure generation module and is used to conduct pre-merging net gain verification and post-merging side effect verification for each candidate search skill document in sequence, and complete the screening by combining local repair benefits and global regression risks. The skill deployment module is connected to the two-stage filtering module and is used to output the final search skill document after filtering, so as to complete the iterative evolution of the agent's skills.
[0020] Compared with the prior art, the present invention has the following beneficial effects: This invention iterates only on external search skill documents throughout the entire process, without requiring training of the underlying language model or modification of the retrieval components. It is adaptable to various commercial deployment scenarios where the model is locked, making it easy to implement. By decomposing and independently repairing abnormal behaviors according to four dimensions—problem analysis, search path, query construction, and stop judgment timing—it effectively avoids rule conflicts caused by mixed repairs of errors from different causes, resulting in stable repair effects. At the same time, it adopts a two-stage screening mechanism of pre-merging single-domain verification followed by post-merging global verification, which can effectively control the side effects brought by new rules. This invention can also optimize the search decision logic, significantly reduce invalid retrieval and timeout issues, greatly reduce search overhead, and the evolved search skills have excellent cross-domain transfer capabilities. In tests on multiple public evaluation sets, the overall quality of the answers has been significantly improved. Attached Figure Description
[0021] Figure 1 A flowchart illustrating the skill evolution of invention; Figure 2 This is a diagram of the overall system architecture for the invention. Detailed Implementation
[0022] The present invention will be further described in detail below with reference to specific embodiments. These descriptions are for explanation purposes only and are not intended to limit the scope of the invention.
[0023] Example 1 refer to Figure 1 As shown, this embodiment presents a skill evolution method for autonomous search agents, assuming... For autonomous search intelligent agent systems, For large language models, For the retrieval tool, For skills documentation, For the question The correct answer. The core optimization objective of this method can be defined as: Among the problems By a fixed model Fixed search engine and documents Joint processing, only allowed Things change, so the goal is to remain unchanged. and Under the premise of basic skills Evolved a more reliable search control skill ; This indicates finding the parameter with the maximum value. Expressing expectations, Represents the evolution set.
[0024] In this setting, skills are not implicit capabilities within the model, but rather an externally controlled document. At runtime, the system injects the skill document, using it to constrain problem parsing, search paths, query construction, and stopping points.
[0025] Specifically, the following steps are included: Step 1: Based on the initial search skill document, drive the autonomous search agent to perform retrieval-enhanced question-answering tasks on the target question set, generating an operation log; for each question... The execution trajectory of a retrieval-enhanced question-answering system can be represented as follows: in, That's the standard answer. It is system output. and They represent the first Wheel movement and retrieval observation, It is the number of execution rounds. Record whether the execution ended successfully, whether the maximum number of rounds was triggered, and other statuses.
[0026] Step 2: Based on the evaluation of the operation logs, divide the samples into normal and abnormal samples. Further decompose the abnormal samples into different problem domains; the set of abnormal samples is represented as follows: in, Represents a set of abnormal samples. For sample index, For the first The execution trajectory of each problem This is the function for evaluating the answer; 0 indicates that the evaluation result is a failure.
[0027] For each abnormal execution record, it is categorized into four independent problem domains based on the characteristics of the erroneous behavior stage: Domain 1 is the problem parsing deviation domain, which mainly includes incorrect target entity positioning, missing constraints, and attribute identification deviation; Domain 2 is the inappropriate search path domain, which mainly includes unreasonable retrieval order, incorrect information source selection, and inefficient multi-hop path planning; Domain 3 is the query construction defect domain, which mainly includes inappropriate query term granularity, unresolved ambiguity, and rewriting direction deviating from the original meaning; Domain 4 is the misalignment of termination timing domain, which mainly includes premature termination due to insufficient evidence and continued invalid retrieval despite sufficient evidence.
[0028] The formula for decomposing outlier samples into domains is expressed as follows: in, For the set of problem domains, To be included in the domain A subset of abnormal behavior records.
[0029] Step 3: For each subset of abnormal samples corresponding to a problem domain, independently generate local revision measures that only apply to that problem domain. Apply the local revision measures to the initial search skill document to generate candidate search skill documents for each problem domain. Each revision measure only modifies the part of the initial search skill document related to that problem domain and does not modify other search control rules across domains.
[0030] This independent domain-based revision has three benefits. First, it maintains the locality of the changes, ensuring that each candidate revision corresponds to a clear problem domain. Second, it improves interpretability because differences in candidate skill documents can be traced back to specific problem domains and failure samples. Third, it provides comparable candidate units for subsequent two-stage constraint screening: the system can determine whether a revision truly fixes its source failure on a problem domain-by-problem-domain basis, rather than simply looking at the overall performance of the final skill.
[0031] It is important to emphasize that the generation of candidate search skills is only an exploratory phase and does not mean that the search skill will be included in the final skill document. This is because each problem domain may generate seemingly reasonable modifications, but these modifications may have too little benefit, too narrow an applicability, or introduce new degradation on other samples.
[0032] Step 4: For each candidate search skill document, a two-stage constraint screening mechanism consisting of pre-merging net gain verification and post-merging side effect testing is adopted. At the same time, the benefits of local repair and the risks of global regression are evaluated to complete the skill evolution and output the final search skill document.
[0033] The first phase of the pre-merging net gain verification process includes: loading each candidate revision measure into the autonomous search agent system separately, rerunning it on the anomalous sample subset of its source problem domain, and statistically analyzing the repair rate; This measure is eligible to proceed to the next stage only when the repair ratio reaches a preset entry threshold. The determination formula is as follows: in, For the set of samples that change from negative to positive in this domain, This is the threshold for the first-stage repair rate.
[0034] The second phase of post-concurrent side effect testing includes: All candidate revisions that passed the pre-merged net gain check were programmatically merged into the initial search skills document and fully validated on the complete sample set; the merging method is represented as follows: in, This indicates the merged skills document. This refers to the initial basic skills document. This indicates a programmatic concatenation operation. This represents the set of problem domain indexes that have been pre-merged and verified. Representing the problem domain The revision measures.
[0035] The total number of repaired samples and the total number of degraded samples after merging are statistically analyzed. Only when the ratio of the number of repaired samples to the number of degraded samples reaches a preset safety threshold is the merged version officially accepted as the final evolved search skill document; the judgment formula is: in, This represents the total number of repaired samples. The total number of degraded samples, It is a very small positive number, used to prevent the denominator from being zero. This is the second-stage safety threshold.
[0036] The output is the final search skill document, which has undergone three processes: domain decomposition, independent customization, and two-stage constraint screening. This document can directly replace the initial search skill document in the autonomous search agent system without changing the model parameters or retrieval configuration.
[0037] Search skill evolution based on the 2Wiki dataset. Using the 2Wiki multi-hop question-answer set as the initial evolution dataset, the initial search skill documents were deployed and run once. Outlier samples were extracted from the results and categorized into four question domains. Candidate revision measures were generated independently for each domain and then subjected to a two-stage constraint screening process to produce the evolved search skill documents. On the 2Wiki hold-out test set, the Qwen3.5-397B-A17B model showed a 3.6 percentage point improvement in Exact Match and a 4.0 percentage point improvement in F1; the Qwen3.5-35B-A3B model showed a 3.6 percentage point improvement in Exact Match and a 3.0 percentage point improvement in F1.
[0038] Out-of-domain transfer validation: Search skill documents evolved from the 2Wiki set were directly applied to evaluation sets such as PopQA, NQ, HotpotQA, and MuSiQue, which did not participate in the evolution. The Qwen3.5-397B-A17B model showed an average ExactMatch improvement of 3.2 percentage points and an F1 improvement of 3.5 percentage points out of domain. The number of timeout termination cases decreased by 71 in the PopQA set, a reduction of 68.3%; and by 72 in the MuSiQue set, a reduction of 65.5%.
[0039] Recommended parameter values: First-stage repair rate threshold τ = 0.02, second-stage safety threshold λ = 1.5, and the number of problem domains is 4. These are recommended values based on experimental verification and are not intended to limit the scope of the claims.
[0040] In addition to the above embodiments, the present invention can also adaptively configure the number of problem domain divisions, the granularity of each domain revision, the first-stage repair rate threshold τ, and the second-stage safety threshold λ according to the model size, retrieval data source type, task difficulty, and search budget of different autonomous search agents. However, its basic process includes domain-based attribution of abnormal execution trajectories, generation of local revision measures, and two-stage constraint screening.
[0041] The evolved search skill document generated by this invention can be deployed in autonomous search agent scenarios such as open-domain question answering, multi-hop reasoning question answering, enterprise knowledge base retrieval, intelligence analysis, and complex information collection, to improve search accuracy, execution stability, and cost controllability without changing the underlying model parameters.
[0042] Example 2 refer to Figure 2 As shown in the figure, this embodiment provides a skill evolution system for autonomous search agents, including a baseline operation log module, an anomaly domain processing module, a candidate measure generation module, a two-stage screening module, and a skill deployment module; The baseline operation log module is used to drive the autonomous search agent to perform retrieval-enhanced question-answering tasks on the target question set based on the initial search skill document, and to collect and generate operation logs; The anomaly domain processing module is connected to the baseline operation log module. It is used to complete the effect evaluation based on the operation log, divide normal samples and abnormal samples, and decompose the abnormal samples into multiple independent problem domains. The candidate measure generation module, connected to the anomaly domain processing module, is used to independently generate local revision measures that only apply to the corresponding problem domain for each subset of anomaly samples, and to adapt the local revision measures to the initial search skill document to generate candidate search skill documents for each problem domain. The two-stage screening module, connected to the candidate measure generation module, is used to sequentially perform pre-merging net gain verification and post-merging side effect verification on each candidate search skill document, and complete the screening by combining local repair benefits and global regression risks. The skill deployment module, connected to the two-stage filtering module, is used to output the final search skill documents after filtering, thus completing the iterative evolution of the agent's skills.
Claims
1. A skill evolution method for autonomous search agents, characterized in that, Includes the following steps: Step 1: Based on the initial search skills document, drive the autonomous search agent to perform retrieval-enhanced question-answering tasks on the target question set and generate an operation log; Step 2: Based on the evaluation of the operation log, divide the samples into normal samples and abnormal samples, and decompose the abnormal samples into different problem domains. Step 3: For each subset of abnormal samples corresponding to a problem domain, independently generate local revision measures that only apply to this problem domain, and apply the local revision measures to the initial search skill documents to generate candidate search skill documents for each problem domain. Step 4: For each candidate search skill document, a two-stage constraint screening mechanism consisting of pre-merging net gain verification and post-merging side effect testing is adopted. At the same time, the benefits of local repair and the risks of global regression are evaluated to complete the skill evolution and output the final search skill document.
2. The skill evolution method for autonomous search agents according to claim 1, characterized in that, In step one, for each problem The execution trajectory of a retrieval-enhanced question-answering system can be represented as follows: in, That's the standard answer. It is system output. and They represent the first Wheel movement and retrieval observation, It is the number of execution rounds. Record whether the execution ended successfully, whether the maximum number of rounds was triggered, and other statuses.
3. The skill evolution method for autonomous search agents according to claim 1, characterized in that, The abnormal sample set in step two is represented as follows: in, Represents a set of abnormal samples. For sample index, For the first The execution trajectory of each problem This is the function for evaluating the answer; 0 indicates that the evaluation result is a failure.
4. The skill evolution method for autonomous search agents according to claim 1, characterized in that, The problem domain mentioned in step two includes the problem parsing bias domain, the search path misconception domain, the query construction defect domain, and the stop judgment timing misalignment domain; The formula for decomposing outlier samples into domains is expressed as follows: in, For the set of problem domains, To be included in the domain A subset of abnormal behavior records.
5. The skill evolution method for autonomous search agents according to claim 1, characterized in that, Each revision measure described in step three only modifies the part of the initial search skills document related to that problem domain, without changing other search control rules across domains.
6. The skill evolution method for autonomous search agents according to claim 1, characterized in that, The pre-merging net gain verification process described in step four includes: loading each candidate revision measure separately into the autonomous search agent system, rerunning it on a subset of anomalous samples from its source problem domain, and statistically analyzing the repair rate; This measure is eligible to proceed to the next stage only when the repair ratio reaches a preset entry threshold. The determination formula is as follows: in, For the set of samples that change from negative to positive in this domain, This is the threshold for the first-stage repair rate.
7. The skill evolution method for autonomous search agents according to claim 1, characterized in that, The post-combination side effect testing process described in step four includes: All candidate revisions that passed the pre-merged net gain check were programmatically merged into the initial search skills document and fully validated on the complete sample set; the merging method is represented as follows: in, For the merged skills documents, This is an initial basic skills document. For programmed splicing operations, For the set of problem domain indexes that have passed the pre-merge verification, For the problem domain The revision measures. The total number of repaired samples and the total number of degraded samples after merging are statistically analyzed. Only when the ratio of the number of repaired samples to the number of degraded samples reaches a preset safety threshold is the merged version officially accepted as the final evolved search skill document; the judgment formula is: in, This represents the total number of repaired samples. This represents the total number of degraded samples. It is a very small positive number, used to prevent the denominator from being zero. This is the second-stage safety threshold.
8. A system for performing the method of any one of claims 1 to 7, characterized in that, It includes a baseline operation log module, an anomaly domain handling module, a candidate measure generation module, a two-stage screening module, and a skills deployment module; The baseline operation log module is used to drive the autonomous search agent to perform retrieval-enhanced question-answering tasks on the target question set based on the initial search skill document, and to collect and generate operation logs; The anomaly domain processing module is connected to the baseline operation log module. It is used to complete the effect evaluation based on the operation log, divide normal samples and abnormal samples, and decompose the abnormal samples into multiple independent problem domains. The candidate measure generation module is connected to the anomaly domain processing module. It is used to independently generate local revision measures that only apply to the corresponding problem domain for each subset of anomaly samples, and to adapt the local revision measures to the initial search skill document to generate candidate search skill documents for each problem domain. The two-stage screening module is connected to the candidate measure generation module and is used to conduct pre-merging net gain verification and post-merging side effect verification for each candidate search skill document in sequence, and complete the screening by combining local repair benefits and global regression risks. The skill deployment module is connected to the two-stage filtering module and is used to output the final search skill document after filtering, so as to complete the iterative evolution of the agent's skills.
Citation Information
Patent Citations
Model reinforcement learning training method and device, equipment and medium
CN121902950A
Retrieval method and system and electronic equipment
CN122173690A
Automatic construction method and system of large language model agent
CN122174861A
