An agent dynamic experience refining and self-evolution method and system based on text gradient driving

Abstract

The application discloses an agent dynamic experience refining and self-evolution method and system based on a text gradient drive, and belongs to the technical field of agent systems, and comprises the following steps: constructing a structured experience unit; abstracting an experience processing process into an operator, and constructing an experience refining pipeline; an agent executes a task based on an adapted experience text and generates an evaluation record; defining an optimizable paragraph in a prompt word template as a text gradient variable, constructing a chain text gradient calculation graph with the text gradient variable as a precursor and a loss variable as output; generating a text gradient feedback in a reverse topological order along the calculation graph through a large language model, and iteratively updating the text gradient variable to realize automatic optimization of a prompt strategy; and driving dynamic experience maintenance based on a frequency threshold and an utility threshold, and automatically eliminating low-value experiences. Through a closed-loop mechanism of task execution, experience refining, text gradient optimization and utility refining, the application enables the agent to continuously self-evolve with task execution without manual intervention.

Description

Technical Field

[0001] This invention belongs to the fields of large language models, intelligent agent systems, experience memory management, text gradient optimization and autonomous evolution technology, specifically involving an intelligent agent dynamic experience refinement and self-evolution method and system based on text gradient driven. Background Technology

[0002] Large language model intelligent agents have become a core development direction in the field of artificial intelligence. By simulating human reasoning, decision-making, and tool-using abilities, they can autonomously complete complex tasks. As application scenarios continue to expand, intelligent agents need to possess the ability to continuously learn from historical execution experience and evolve autonomously to adapt to dynamically changing task requirements.

[0003] Existing intelligent agent technology suffers from three major bottlenecks: First, the memory mechanism is static. Traditional memory enhancement methods only achieve indiscriminate storage and simple retrieval of the original dialogue trajectory, without structured refinement and dynamic optimization of experience, resulting in high experience noise, low reuse rate, and poor generalization ability. Second, the optimization process is manual. Prompt words and strategy adjustments rely on human experience, which is inefficient, highly subjective, and cannot utilize task feedback to achieve automated iteration. Third, the evolutionary link is fragmented. Experience refinement, strategy optimization, and memory maintenance are independent of each other and do not form a complete closed loop, causing the intelligent agent's capabilities to fail to continuously improve with task execution.

[0004] In recent years, related technologies have made some progress: the ReMe framework proposed the idea of ​​experience distillation and context reuse, realizing the transformation from raw trajectories to structured experience, but it lacks a unified feedback optimization link, and the quality of experience depends on manual filtering; TextGrad implemented backpropagation of text gradients based on natural language feedback, providing the possibility of automated optimization of text-type variables, but it is only used for fine-tuning prompt words and is not deeply integrated with the dynamic experience pool, so it cannot achieve the co-evolution of strategies and experience. None of the above technologies have formed a complete closed loop of "execution-distillation-reuse-optimization-refinement", making it difficult to support the long-term, stable, and autonomous evolution of intelligent agents in complex and dynamic scenarios. Summary of the Invention

[0005] This invention addresses the core problems of static experience, manual optimization, and fragmented evolution in existing technologies. It provides a text gradient-driven method for dynamic experience refinement and self-evolution of intelligent agents. The method abstracts the experience processing into standardized operators, constructing a scalable operator-based experience refinement pipeline. It defines standardized MemoryNode structured experience units to achieve unique identification, vector retrieval, and version management of experiences. It abstracts optimizable paragraphs of prompt word templates into text gradient variables, constructing a chain-like computation graph of "prompt variable - response variable - loss variable." A dual-engine large language model is used to generate text gradients and update variables respectively. A multi-dimensional utility evaluation system based on retrieval frequency and contribution success rate is established to dynamically maintain a high-quality experience pool. Ultimately, this forms a fully autonomous closed loop of experience refinement, reuse, optimization, and updating, enabling the intelligent agent to continuously improve its capabilities without human intervention during task execution. This method is applicable to various scenarios such as general intelligent agents, tool invocation, task planning, and memory-enhanced question answering.

[0006] According to one aspect of this specification, a method for dynamic experience refinement and self-evolution of an intelligent agent based on text gradient-driven approaches is provided, comprising:

[0007] Step 1: Initialize the agent system, including: constructing the structured experience units of the agent system, initializing the basic cue variables and the text gradient optimizer;

[0008] Step 2: Abstract the experience processing of the intelligent agent system into standardized operators, and construct an experience refinement pipeline including query construction operators, experience retrieval operators, reordering operators, and experience rewriting operators; process the current task context through the experience refinement pipeline to obtain experience text adapted to the current task context;

[0009] Step 3: Execute the current task based on the experience text adapted to the current task context, generate the execution trajectory, score the task output after execution, and generate an evaluation record;

[0010] Step 4: Map the initialized basic cue variables to cue variables in the text gradient variables, combine the response variables and loss variables to form text gradient variables, construct a chained text gradient computation graph based on the text gradient variables, generate a gradient list of text gradients stored in the current node, update the cue variables according to the text gradients, output the updated cue variables, and write the updated cue variables back to the cue word template corresponding to the basic cue variables, so that they take effect when the next round of experience processing is initialized;

[0011] Step 5: Calculate the utility value of the experience and eliminate low-value experiences based on their utility value.

[0012] Step 6: Repeat steps 2 to 5 to allow the agent to continuously evolve as the task is executed without human intervention until the convergence condition is met.

[0013] As a further technical solution, the working process of the experience-refined pipeline includes:

[0014] Receive the current task context, generate a structured query question, and generate a query vector based on the structured query question;

[0015] Based on the query vector, retrieve K relevant experience nodes from the vector database;

[0016] The relevant experience nodes are semantically reordered using a large language model. A comprehensive score is calculated based on the confidence score and verification score of the semantically reordered experience nodes. Experience nodes with a comprehensive score lower than a set score threshold are filtered out to obtain candidate experience nodes.

[0017] Based on candidate experience nodes, combined with the current query request and recent dialogue messages, formatted rendering is performed to generate experience text that adapts to the current task context.

[0018] As a further technical solution, the chained text gradient computation graph includes a three-layer structure, specifically:

[0019] The cue word variable layer contains at least one cue variable, and the differentiability flag of all cue variables is true;

[0020] The response variable layer contains an intermediate node whose value is the concatenated text of the current prompt variable and the task performance analysis context;

[0021] The loss variable layer contains an output node, the value of which includes a loss value and a text string describing the optimization objective.

[0022] As a further technical solution, text gradients are generated, including traversing each node in reverse topological order based on a chained text gradient computation graph, finding nodes whose differentiability flag is true, and performing the following operations:

[0023] Based on the nodes with differentiability flags set to true, find the nodes that are predecessors of the current node and obtain the non-empty gradient text. Concatenate the gradient text to obtain the entire text and use it as the downstream gradient.

[0024] Obtain system prompts and input text from the predecessor node of the current node, obtain response text from the successor node of the current node, and construct a dialogue context based on the response text, system prompts, and input text;

[0025] The backpropagation prompts are constructed based on the downstream gradient and the dialogue context, and then input into the first language model to generate text feedback, which serves as the text gradient.

[0026] As a further technical solution, the cue variables are updated according to the text gradient, including performing the following operations for each cue variable whose gradient list is not empty:

[0027] Construct optimizer cue words, including current variable values, role descriptions, all cumulative text gradient feedback, and output format instructions;

[0028] Input the optimizer prompts into the second largest language model and output the complete text content;

[0029] Based on the preset beginning and end tags, the complete text content is matched, truncated and removed to obtain the optimized variable text value;

[0030] Update the current prompt variable based on the text value of the optimization variable, clear the gradient list corresponding to the current prompt variable, and output the updated prompt variable.

[0031] As a further technical solution, the expression for the loss variable is:

[0032]

[0033] in, Represents the loss variable. Let be the average accuracy of the most recent N tasks.

[0034] As a further technical solution, the structured experience unit includes the following fields: experience identifier, experience type, associated topic, applicable conditions, content text, message time, citation identifier, creation time, modification time, source author, related rating, vector representation, and extended data.

[0035] As a further technical solution, the text gradient variable includes five attributes: text value, role description, differentiability flag, gradient list, and predecessor list.

[0036] According to one aspect of this specification, a text gradient-driven intelligent agent dynamic experience refinement and self-evolution system is provided, comprising:

[0037] The first processing module is used to initialize the agent system, including: constructing the structured experience units of the agent system, initializing basic cue variables and text gradient optimizer;

[0038] The second processing module is used to abstract the experience processing process of the intelligent agent system into standardized operators, and construct an experience refinement pipeline including query construction operators, experience retrieval operators, reordering operators and experience rewriting operators; and process the current task context through the experience refinement pipeline to obtain experience text adapted to the current task context.

[0039] The third processing module is used to execute the current task based on the experience text adapted to the current task context, generate the execution trajectory, score the task output after execution, and generate an evaluation record.

[0040] The fourth processing module is used to map the initialized basic cue variables to cue variables in the text gradient variables, combine the response variables and loss variables to form text gradient variables, construct a chained text gradient computation graph based on the text gradient variables, generate a gradient list of text gradients stored in the current node, update the cue variables according to the text gradients, output the updated cue variables, and write the updated cue variables back to the cue word template corresponding to the basic cue variables, so that they take effect when the next round of experience processing is initialized.

[0041] The fifth processing module is used to calculate the utility value of experience and eliminate low-value experience based on the size of the utility value.

[0042] The sixth processing module is used to repeatedly execute the second to fifth processing modules, so that the agent can continuously evolve as the task is executed without human intervention until the convergence condition is met.

[0043] According to one aspect of the present invention, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the text gradient-driven agent dynamic experience refinement and self-evolution method.

[0044] Compared with the prior art, the beneficial effects of the present invention are:

[0045] 1. This invention proposes a standardized operator system with a unified base class and chained arrangement by designing an operator-based pipeline architecture. It modularly decomposes and combines the entire processing flow, such as experience query, retrieval, reordering, and rewriting, to achieve the generalization, scalability, and pluggable configuration of experience processing logic. This improves the overall reusability and scenario adaptability of the pipeline and reduces the workload of secondary development and modification of experience processes under different task scenarios.

[0046] 2. This invention constructs a chain-like text gradient computation graph and proposes a text gradient backpropagation mechanism, which transforms the task execution effect into iterative text gradient feedback. This enables fully automatic, manual-free optimization of the prompt word strategy, improves the iteration efficiency and dynamic adaptation capability of the prompt strategy, and reduces the subjectivity and time cost of traditional manual adjustment of prompt words.

[0047] 3. This invention proposes a dual-engine large model collaborative optimization mechanism, which divides the backpropagation engine into a variable optimization engine and decouples the text gradient generation and text variable update. This achieves accurate gradient feedback generation and targeted optimization of prompt text, improves the stability and optimization effect of text gradient optimization, and avoids the output deviation and performance loss caused by a single model undertaking multiple tasks at the same time.

[0048] 4. This invention designs a closed-loop self-evolutionary process that connects experience refinement, task execution, effect evaluation, gradient optimization, and experience maintenance. The data generated by a single task execution is fed back to the experience base and prompting strategy, realizing the transformation of the agent from passive execution to active and continuous learning. This improves the agent's generalization ability and long-term performance in dynamic task scenarios and solves the core pain points of fragmented evolutionary links and the inability to continuously improve capabilities in traditional technologies. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 The flowchart illustrates the text gradient-driven dynamic experience refinement and self-evolution method for intelligent agents provided in this embodiment of the invention.

[0051] Figure 2 This is a flowchart of the operator-based empirical refining pipeline according to an embodiment of the present invention.

[0052] Figure 3 This is a schematic diagram illustrating the chained text gradient calculation and update in an embodiment of the present invention.

[0053] Figure 4 This is a schematic diagram of the utility-driven dynamic experience maintenance mechanism in an embodiment of the present invention. Detailed Implementation

[0054] It should be noted that: an intelligent agent is an interactive execution subject that interacts with users and performs various business tasks;

[0055] The intelligent agent system is a complete, integrated operating system comprising multiple components, including an interactive intelligent agent, an experience refinement pipeline, a vector database, an evaluation module, a text gradient optimization module, and an experience pool maintenance module. All self-evolution, experience management, and gradient optimization processes in this application are implemented using the intelligent agent system. Unless otherwise specified below, all operators, evaluations, gradient updates, and experience maintenance operations operate within the intelligent agent system.

[0056] Experience nodes are the smallest structured storage units in the agent's experience pool and vector database. As data entities containing experience text, time information, vector embedding, utility value, and historical version information, all retrieval, reordering, version management, and utility evaluation operations are processed using experience nodes.

[0057] Experience refers to the business text content and knowledge information carried within an experience node, and is the core data content within the experience node.

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] like Figure 1 As shown, this embodiment of the invention proposes a text gradient-driven method for dynamic experience refinement and self-evolution of an intelligent agent, including: Step 1, initializing the intelligent agent system, including: constructing structured experience units of the intelligent agent system, initializing basic cue variables, and a text gradient optimizer; Step 2, abstracting the experience processing of the intelligent agent system into standardized operators, constructing an experience refinement pipeline including query construction operators, experience retrieval operators, reordering operators, and experience rewriting operators; processing the current task context through the experience refinement pipeline to obtain experience text adapted to the current task context; Step 3, executing the current task based on the experience text adapted to the current task context, generating an execution trajectory, scoring the task output after execution, and generating... Evaluation Record; Step 4: Map the initialized basic cue variables to cue variables in the text gradient variables, combine them with the response variable and loss variable to form text gradient variables, construct a chained text gradient computation graph based on the text gradient variables, and generate a gradient list for the current node to store the text gradients. Update the cue variables according to the text gradients, output the updated cue variables, and write the updated cue variables back to the cue word template corresponding to the basic cue variables. This will take effect during the initialization of the next round of experience processing. Step 5: Calculate the utility value of the experience, and filter and eliminate low-value experiences based on the size of the utility value. Step 6: Repeat steps 2 to 5 to allow the agent to continuously self-evolve as the task is executed without human intervention until the convergence condition is met. The convergence condition is that after N consecutive rounds of backpropagation iterations, there are no new gradient records in the gradient lists corresponding to all text gradient variables, and the backpropagation engine no longer generates new gradient feedback; where N is a preset convergence judgment round threshold.

[0060] Specifically, step 1 includes the following:

[0061] 1. Construct structured experience units for the intelligent agent system and persistently store experience data. The structured experience unit adopts the MemoryNode data structure, containing the following 13 fields: Experience Identifier (memory_id): A unique identifier generated based on the content SHA-256 hash (taking the first 16 bits); Experience Type (memory_type): An enumeration of memory types (e.g., SUMMARY, PERSONAL, etc.); Associated Topic (memory_target): The target or topic associated with this experience; Applicable Conditions (when_to_use): A description of the applicable scenario conditions, used as a matching basis during vector retrieval; Content Text (content): The actual content text of the experience; Message Time (message_time): The timestamp of the message that generated the experience; Reference Identifier (ref_memory_id): The original historical memory reference ID associated with the experience; Creation Time (time_created): The experience creation time; Modification Time (time_modified): The experience's last modification time; Source Author (author): The author or source of the experience; Relevance Rating (score): The relevance or importance rating; Vector Representation (vector): The vector embedding representation of the experience content; Extended Data (metadata): An extended metadata dictionary that supports flexible expansion. When the content text or applicable conditions field of the experience is modified, the modification timestamp is automatically updated; when the content text is modified, the experience identifier is automatically recalculated based on the content hash value.

[0062] 2. Initialize the basic prompt variables of the intelligent agent system, including optimized text paragraphs such as role definitions, general task instructions, inference guidance templates, and output format constraints. The basic prompt variables are text paragraphs that support automatic optimization in the prompt word templates used in the experience refinement pipeline, and belong to the original text objects that can be optimized.

[0063] 3. Initialize the text gradient optimizer of the intelligent agent system and set the default maximum number of iterations to 3.

[0064] Specifically, step 2 includes the following:

[0065] The experience processing of the agent is abstracted into standardized operators. All operators used in the experience refinement pipeline inherit from the unified base class `BaseOp`, providing context management, prompt word template rendering, large language model invocation, retry and exception handling (default maximum of 1 retries), and vector storage access functionality. (See reference...) Figure 2The experience refinement pipeline uses a chained combination of operators to achieve chained arrangement of operators, consisting of the following four operators in sequence: query construction operator → experience retrieval operator (default retrieval count is 5) → reordering operator → experience rewriting operator. Specifically, the query construction operator first generates a query vector based on the user query; then, the experience retrieval operator retrieves relevant experience from the vector database; subsequently, the reordering operator performs fine-grained sorting of the relevant experience nodes; and finally, the experience rewriting operator generates contextual experience text adapted to the current task. The vector database construction process is as follows: when the agent system creates and updates each MemoryNode experience unit, it generates a corresponding vector embedding and stores the "experience data + vector" uniformly in the vector database. The specific working process of the experience refinement pipeline is as follows:

[0066] Step 2.1, Query construction.

[0067] Receive the current task context, including the user's original query request (covering forms such as question answering, tool calls, task planning, and other common agent tasks) and recent dialogue history. Generate a structured query question by integrating the current task context information, and generate a query vector based on the structured query question. Write the structured query question and query vector together into the context metadata.

[0068] Step 2.2, Experience Search.

[0069] Based on the query vector, the top-K (default K=5) relevant experience nodes are retrieved from the vector database. The relevant experience nodes are deduplicated by content text field, and relevance scores are recorded by relevance rating field. Relevant experience nodes are retained based on their relevance scores. Optionally, this embodiment of the invention can also filter by a minimum score threshold, retaining only experience nodes with scores higher than the minimum score threshold. The retained relevant experience nodes are written into the experience node list field in the context metadata to provide standard input data for the next-level reordering operator, enabling data flow and context sharing between operators, and ensuring the serial execution of the experience refinement pipeline. Specifically, when retrieving relevant experience nodes from the vector database, cosine similarity is used to calculate the distance between the query vector and the experience node vector:

[0070]

[0071] in, For query vector, The query vector is the experience vector. Cosine similarity represents the semantic similarity between the query vector and each experience node. The closer the value is to 1, the higher the semantic and scenario matching degree between the two.

[0072] Step 2.3, Experience Reranking (RerankMemory). Read the experience node list field from the context metadata and perform two levels of fine-grained processing sequentially:

[0073] 1. LLM Reordering (optional): Call the large language model to perform semantic reordering on the list of experience nodes and generate reordered candidate experience nodes;

[0074] 2. Score Filtering (Optional): Calculate a comprehensive score based on the confidence score and validation score of the reordered candidate experience nodes. The calculation formula is as follows:

[0075] (2)

[0076] Filter out low-quality candidate experience nodes whose overall score is below the minimum score threshold (default 0.3) and write the filtered candidate experience nodes back to the context metadata.

[0077] Step 2.4, Experience Rewriting and Context Adaptation (RewriteMemory).

[0078] The system reads the candidate experience nodes from the context metadata, reorders and filters low-quality candidate experience nodes, and then performs formatted rendering based on the current task context and recent dialogue history (the last 3 entries). This generates experience text adapted to the current task context, including the following two modes:

[0079] 1. Direct formatting mode: The applicable conditions and content of each experience node are formatted into structured text (e.g., "Experience 1: Applicable conditions: xxx, Content: xxx"), without semantic rewriting;

[0080] 2. LLM Rewrite Mode (Optional): This mode calls the Large Language Model to rewrite general experience into experience text adapted to the current task context. If an exception occurs in the rewrite mode, it automatically degrades to the direct formatting mode. The general experience consists of the basic experience content within the original experience nodes after retrieval and reordering. This content is highly generic and lacks customization tailored to the specific user query and dialogue scenario, resulting in poor adaptability. Therefore, it requires LLM rewriting.

[0081] Write the experience text adapted to the current task context into the answer field of the context response section in the context metadata for use by the agent in task execution.

[0082] Specifically, step 3 includes the following:

[0083] The agent executes the current task based on the experience text adapted to the current task context, and generates an execution trajectory. The execution trajectory is recorded in the form of a multi-turn dialogue history, including the user input and the response of the large language model in each turn. After the execution is completed, the agent's evaluation module is called to score the task output and generate an evaluation record, which provides a basis for subsequent loss calculation and text gradient backpropagation. The task output is the user reply text generated by the agent in combination with the context and adaptation experience. The evaluation module receives three types of input: task requirements, dialogue execution trajectory, and adaptation experience, and executes two types of scoring simultaneously: (1) Classification scoring: The evaluation model determines the category of the output text, which is divided into three categories: correct, illusion, and omission. Correct means that the reply is completely in line with the task requirements, illusion means that there is false and fabricated content, and omission means that the core task information is missing; (2) Continuous value scoring: Based on the weighted calculation of the proportion of illusion, the degree of information omission, and semantic matching deviation, the output value range is 0 to 1. The higher the score, the better the quality of the task output. The evaluation module encapsulates the classification label, continuous score, and text defect description into an evaluation record, which is used to construct the loss variable in the computation graph.

[0084] Specifically, refer to Figure 3 Step 4 includes the following:

[0085] Step 4.1: Define and initialize text gradient variables

[0086] Text gradient variables are the basic node units of the computation graph. Defining the data structure and five attributes of text gradient variables, and completing variable definition and instantiation, are prerequisites for building the computation graph. Each variable contains five attributes: the first attribute is the text value, storing the current text content of the variable; the second attribute is the role description, explaining the function of the variable in the system in natural language, used to guide subsequent gradient generation and parameter updates; the third attribute is the differentiability flag, indicating whether the variable needs to have its gradient calculated; the fourth attribute is the gradient list, accumulating the text-based gradient feedback generated by the large language model; and the fifth attribute is the predecessor list, recording all predecessor nodes of the variable in the computation graph, used for topology traversal during backpropagation.

[0087] The initialized basic cue variables are mapped to cue variables in the text gradient variables, serving as precursor nodes in the computation graph. The response variable is dynamically generated during runtime, created in real-time as the experience refinement pipeline executes tasks and calls the large model to obtain output. It stores intermediate execution results such as model outputs and rewritten experience text from each round, and is automatically added to the text gradient variable set, serving as an intermediate node in the computation graph. The loss variable is generated from the evaluation records. After step 3 is completed, the evaluation records are read, and failure cases such as illusions and omissions are converted into quantified loss text, dynamically constructing the loss variable, which serves as the final output node in the computation graph. The text gradient variables are then constructed based on the cue variables, response variables, and loss variables.

[0088] Step 4.2: Construct a chained text gradient computation graph

[0089] A chained text gradient computation graph is constructed based on text gradient variables. This chained text gradient computation graph comprises a three-layer structure, with the cue variable as the predecessor node, the response variable as the intermediate node, and the loss variable as the output node, as detailed below:

[0090] The first layer is the cue variable layer, which contains at least one cue variable, and all cue variables are differentiable if true. Cue variables are instantiated from optimizable paragraphs in the cue word template. Typical optimizable paragraphs include context priority rule paragraphs and output format constraint paragraphs.

[0091] The second layer is the response variable layer, which contains an intermediate node. The value of the intermediate node is the concatenated text of the current prompt variable and the task performance analysis context. The predecessor node is associated with all prompt variables in the first layer.

[0092] The third layer is the loss variable layer, containing one output node. The output node's value includes the loss value and a text string describing the optimization objective, with its predecessor pointing to the response variable node in the second layer. The data flow in the computation graph is: cue word variable → response variable → loss variable. Here, the cue word template represents the original file; the optimizable cue word segment represents the text fragment within the cue word template; and the cue variable represents the gradient node in the computation graph.

[0093] Specifically, an example of a prompt word template is as follows:

[0094] (1) Role positioning

[0095] Imagine you are a text gradient analysis engine, responsible for locating defects in the current prompt text based on downstream gradient feedback and the complete dialogue context, and outputting structured gradient text.

[0096] (2) Downstream gradient feedback (inherited from successor nodes)

[0097] (3) Complete dialogue context

[0098] [Text Excerpt]:

[0099] [User Input]:

[0100] [Model Response]:

[0101] (4) Output requirements

[0102] 1. Analyze each line of the current prompt text to identify the root cause of the problem;

[0103] 2. Provide actionable modification and optimization directions from different dimensions;

[0104] 3. Output gradient feedback text without adding any extra irrelevant descriptions.

[0105] Step 4.3: Construct the loss variable

[0106] 1. Collect all evaluation records for each task, calculate the overall task accuracy, and distinguish between correct and failed samples;

[0107] 2. Sample from the failed samples labeled as hallucinations or omissions, and extract the user questions, agent output answers, standard reference content, and evaluation judgment reasons corresponding to the samples;

[0108] 3. Integrate the various types of information obtained from sampling to generate a failure analysis text;

[0109] 4. Construct loss variables for gradient backpropagation based on failure analysis text. The expression is:

[0110] (3)

[0111] in, Let be the average accuracy of the most recent N tasks.

[0112] Step 4.4: Backpropagation (generating text gradients)

[0113] Based on the chained text gradient computation graph, traverse each node in reverse topological order (referring to the computational nodes for calculating the chained text gradient, used only for gradient backpropagation and generating gradient text) to find nodes with a differentiability flag of true. For each node with a differentiability flag of true, perform the following operations:

[0114] 1. Collect downstream gradients: For each node with a differentiable flag of true, find all nodes that are predecessors of the current node, i.e., successor nodes, obtain their non-empty gradient text, and concatenate them into a downstream gradient string; specifically, each successor node will pass up the gradient text feedback it has already generated; concatenate the multiple gradient texts from multiple successor nodes into a whole text; the concatenated whole text is the downstream gradient that the current node needs to use.

[0115] 2. Constructing the dialogue context: Obtain system prompts and input text from the predecessor node of the current node, obtain response text from the successor node of the current node, and construct the dialogue context based on the system prompts, input text, and response text;

[0116] 3. Constructing Backpropagation Cue Words: Backpropagation cue words are constructed based on the downstream gradient and the dialogue context. This involves filling the template with both the downstream gradient and the dialogue context to obtain complete backpropagation cue words. Specifically, when the downstream gradient is not empty, a chained backpropagation template with a gradient field is selected to construct the backpropagation cue words; when the downstream gradient is empty, a blank basic template is selected to construct the backpropagation cue words.

[0117] 4. Generate text gradients: Input the backpropagation prompts into the first language model (backpropagation engine, which is the qwen3-8b model in this embodiment of the invention), output text feedback, and store it as the text gradient in the gradient list of the current node (a collection structure used to accumulate and store multiple gradient feedbacks, which is a data container).

[0118] Step 4.5: Update the parameters of the cue variables based on the text gradient.

[0119] For each variable whose gradient list is empty, skip the update operation and do not perform any optimization on that variable, leaving the original text content unchanged.

[0120] For each hint variable where the gradient list is not empty, the details are as follows:

[0121] 1. Construct optimizer cue words, including current variable values, role descriptions, all cumulative text gradient feedback, and output format instructions;

[0122] 2. Input the optimizer's suggestions into the second language model and output the complete text content;

[0123] 3. Perform string matching on the complete text content based on the preset beginning and end separator tags, extract the content between the tags, remove redundant text, and obtain the optimized variable text.

[0124] 4. Update the current prompt variable using the optimized variable text, clear the gradient list corresponding to the current text gradient variable, output the updated prompt variable, and write it back to the prompt word template corresponding to the basic prompt variable. This will take effect during the initialization of the next round of experience processing.

[0125] Specifically, step 5 includes the following:

[0126] After multiple rounds of iterative execution of steps 1 to 4, statistical records generated for each experience node are accumulated, such as... Figure 4 As shown, quantitative evaluation and dynamic operation and maintenance are carried out on all structured experience nodes; where "all" refers to all structured experience nodes currently stored in the experience pool of the intelligent agent system, excluding historical experience nodes that have been eliminated or deleted.

[0127] The intelligent agent system tracks statistics such as the number of times each experience is used, the number of successful uses, the number of times it is retrieved, and the creation time. It calculates the utility value of each experience and filters out low-value experiences based on their utility values, continuously purifying the experience pool and providing high-quality data sources for the next round of experience retrieval and refinement. The utility value of the experience... The calculation formula is as follows:

[0128] (4)

[0129] Where success_count is the number of times an experience has been successfully used, referring to the cumulative number of times the experience has been invoked and has helped the agent complete the task and output valid results; use_count is the total number of times an experience has been used, referring to the cumulative total number of times the experience has been formally invoked by the experience refinement pipeline to participate in task processing; The success rate of contributing experience; retrieval_count is the cumulative number of times a single experience has been retrieved; total_retrieval is the cumulative total number of times all experiences in the experience pool have been retrieved; For the frequency of retrieval based on experience, The timeliness factor is decaying monthly. Indicates the time the experience was created. Indicates the current time. Formula (4) has two types of terms: positive benefits (contribution success rate, number of searches, etc.) + negative losses (timeliness factors, etc.). It is not necessary to strictly implement normalization, that is, the sum of the total weights does not need to be 1. If the total weights are forcibly pulled to 1, the negative impact of time decay will be greatly amplified.

[0130] Specifically, the embodiments of the present invention also include an experience version management mechanism: when updating experience content, historical versions are automatically retained, and version backtracking and rollback are supported; after the user manually marks and corrects the experience, the intelligent agent system automatically recalculates the utility value and vector embedding of the experience.

[0131] Specifically, the embodiments of the present invention form a complete closed loop of "task execution → experience refinement → text gradient optimization → utility refinement", including:

[0132] 1. The agent performs tasks based on the current prompting strategy and experience base, and generates evaluation records;

[0133] 2. The experience refinement pipeline extracts and stores experience from task interaction data;

[0134] 3. The text gradient optimization module performs backpropagation optimization on the prompting strategy based on the evaluation results;

[0135] 4. The utility dynamic refinement module regularly maintains the experience base and eliminates low-value experience.

[0136] 5. After multiple iterations, the agent's prompting strategy is continuously optimized and the quality of experience is constantly improved, achieving continuous autonomous evolution of capabilities without human intervention.

[0137] Specifically, after testing, the method proposed in this embodiment of the invention, after multiple rounds of operation on the benchmark dataset Appworld (168 standardized tasks), combined with memory enhancement and text gradient optimization, significantly improved the task completion rate of the agent in tool calling tasks; and effectively reduced the illusion rate and omission rate of answers on the question answering benchmark, thus improving the accuracy of memory-enhanced question answering.

[0138] Table 1. Experimental Results of the Model

[0139]

[0140] As shown in Table 1, for the Qwen3-8B model, after integrating the proposed solution, the pass@1 index increased from 0.1548 to 0.1726, an increase of 1.78%; the best@1 index increased from 0.4793 to 0.4911, an increase of 1.18%. For the DeepSeek-V3.2 model, after incorporating the proposed solution, the pass@1 index increased from 0.6786 to 0.7083, an increase of 2.97%; the best@1 index increased from 0.8554 to 0.8698, an increase of 1.44%. The experimental results of both models demonstrate that the text gradient-driven agent dynamic experience refinement and self-evolution method proposed in this invention can effectively improve the agent's overall ability to complete tasks. By leveraging the operator-based experience refinement pipeline, chained text gradient optimization, and a closed-loop mechanism for utility-driven experience maintenance, the agent can continuously optimize prompting strategies, filter high-quality experiences, and enhance semantic matching and task execution capabilities. This effectively reduces issues such as answer illusions and content omissions, demonstrating significant performance gains in tool invocation and question-and-answer tasks. The practicality and implementation value of the solution have been fully validated.

[0141] The implementation of the various embodiments of this invention is based on programmed processing through a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of this invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of this invention provide a text gradient-driven intelligent agent dynamic experience refinement and self-evolution system. This system is used to execute a text gradient-driven intelligent agent dynamic experience refinement and self-evolution method from the above method embodiments.

[0142] The system includes: a first processing module for initializing the agent system, including constructing structured experience units, initializing basic cue variables, and a text gradient optimizer; a second processing module for abstracting the experience processing of the agent system into standardized operators, constructing an experience refinement pipeline including query construction operators, experience retrieval operators, reordering operators, and experience rewriting operators; processing the current task context through the experience refinement pipeline to obtain experience text adapted to the current task context; a third processing module for executing the current task based on the experience text adapted to the current task context, generating an execution trajectory, scoring the task output after execution, and generating an evaluation record; and a fourth processing module for processing the initialized... The basic cue variables are mapped to cue variables in the text gradient variables. Combined with response variables and loss variables, the text gradient variables are constructed. A chained text gradient computation graph is built based on the text gradient variables, and the text gradient is generated and stored in the gradient list of the current node. The cue variables are updated according to the text gradients, the updated cue variables are output, and the updated cue variables are written back to the cue word template corresponding to the basic cue variables. The updated cue variables take effect when the next round of experience processing is initialized. The fifth processing module is used to calculate the utility value of the experience and eliminate low-value experiences according to the size of the utility value. The sixth processing module is used to repeatedly execute the second to fifth processing modules, so that the agent can continuously self-evolve with the task execution without human intervention until the convergence condition is met.

[0143] This invention provides a text gradient-driven dynamic experience refinement and self-evolution system for intelligent agents. Addressing the core problems of static experience, manual optimization, and fragmented evolution in existing technologies, this method abstracts the experience processing into standardized operators, constructing a scalable operator-based experience refinement pipeline. It defines standardized MemoryNode structured experience units to achieve unique identification, vector retrieval, and version management of experiences. It abstracts optimizable paragraphs of prompt word templates into text gradient variables, constructing a chained computation graph of "prompt variable - response variable - loss variable." A dual-engine large language model is used to generate text gradients and update variables respectively. A multi-dimensional utility evaluation system based on retrieval frequency and contribution success rate is established to dynamically maintain a high-quality experience pool. Ultimately, this forms a fully autonomous closed loop of experience refinement, reuse, optimization, and updating, enabling the intelligent agent to continuously improve its capabilities without human intervention during task execution. This system is applicable to various scenarios such as general intelligent agents, tool invocation, task planning, and memory-enhanced question answering. The dual-engine division in the dual-engine large language model is based on functional responsibilities rather than the weights of the underlying model. The division includes: (1) the reverse engine, which is responsible for receiving the computation graph and generating text gradient feedback; and (2) the optimizer engine, which is responsible for receiving gradients and rewriting the prompt variable text. The two engines perform independent tasks, make independent calls, and have independent input-output links. Even if the underlying model reuses the same base model (such as qwen3-8b), the two independent instances still carry the two types of functions and still meet the definition of "dual-engine". At the same time, the embodiments of the present invention also support the use of two different base models to deploy dual engines.

[0144] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides an electronic device, including a memory and a processor. The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize a text gradient-driven intelligent agent dynamic experience refinement and self-evolution method as proposed in the above embodiments.

[0145] Finally, it should be noted that the above specific embodiments are merely representative examples of the present invention. Obviously, the present invention is not limited to the above specific embodiments and many variations are possible. Any simple modifications, equivalent changes, and alterations made to the above specific embodiments based on the technical essence of the present invention should be considered within the protection scope of the present invention.

Claims

1. A method for dynamic experience refinement and self-evolution of intelligent agents based on text gradient-driven approaches, characterized in that, include: Step 1: Initialize the agent system, including: constructing the structured experience units of the agent system, initializing the basic cue variables and the text gradient optimizer; Step 2: Abstract the experience processing of the intelligent agent system into standardized operators, and construct an experience refinement pipeline including query construction operators, experience retrieval operators, reordering operators, and experience rewriting operators; process the current task context through the experience refinement pipeline to obtain experience text adapted to the current task context; Step 3: Execute the current task based on the experience text adapted to the current task context, generate the execution trajectory, score the task output after execution, and generate an evaluation record; Step 4: Map the initialized basic cue variables to cue variables in the text gradient variables, combine the response variables and loss variables to form text gradient variables, construct a chained text gradient computation graph based on the text gradient variables, generate a gradient list of text gradients stored in the current node, update the cue variables according to the text gradients, output the updated cue variables, and write the updated cue variables back to the cue word template corresponding to the basic cue variables, so that they take effect when the next round of experience processing is initialized; Step 5: Calculate the utility value of the experience and eliminate low-value experiences based on their utility value. Step 6: Repeat steps 2 to 5 to allow the agent to continuously evolve as the task is executed without human intervention until the convergence condition is met.

2. The method for dynamic experience refinement and self-evolution of an agent based on text gradient-driven approach according to claim 1, characterized in that, The working process of the experience-refined pipeline includes: Receive the current task context, generate a structured query question, and generate a query vector based on the structured query question; Based on the query vector, retrieve K relevant experience nodes from the vector database; The relevant experience nodes are semantically reordered using a large language model. A comprehensive score is calculated based on the confidence score and verification score of the semantically reordered experience nodes. Experience nodes with a comprehensive score lower than a set score threshold are filtered out to obtain candidate experience nodes. Based on candidate experience nodes, combined with the current query request and recent dialogue messages, formatted rendering is performed to generate experience text that adapts to the current task context.

3. The method for dynamic experience refinement and self-evolution of an intelligent agent based on text gradient driving according to claim 1, characterized in that, The chain-like text gradient computation graph includes a three-layer structure, specifically: The cue word variable layer contains at least one cue variable, and the differentiability flag of all cue variables is true; The response variable layer contains an intermediate node whose value is the concatenated text of the current prompt variable and the task performance analysis context; The loss variable layer contains an output node, the value of which includes a loss value and a text string describing the optimization objective.

4. The method for dynamic experience refinement and self-evolution of an agent based on text gradient-driven approach according to claim 1, characterized in that, Generate text gradients, including a chain-based text gradient computation graph, traversing nodes in reverse topological order, finding nodes with a differentiable flag of true, and performing the following operations: Based on the nodes with differentiability flags set to true, find the nodes that are predecessors of the current node and obtain the non-empty gradient text. Concatenate the gradient text to obtain the entire text and use it as the downstream gradient. Obtain system prompts and input text from the predecessor node of the current node, obtain response text from the successor node of the current node, and construct a dialogue context based on the response text, system prompts, and input text; The backpropagation prompts are constructed based on the downstream gradient and the dialogue context, and then input into the first language model to generate text feedback, which serves as the text gradient.

5. The method for dynamic experience refinement and self-evolution of an agent based on text gradient-driven approach according to claim 1, characterized in that, The cue variables are updated based on the text gradient, including performing the following operations for each cue variable whose gradient list is not empty: Construct optimizer cue words, including current variable values, role descriptions, all cumulative text gradient feedback, and output format instructions; Input the optimizer prompts into the second largest language model and output the complete text content; Based on the preset beginning and end tags, the complete text content is matched, truncated and removed to obtain the optimized variable text value; Update the current prompt variable based on the text value of the optimization variable, clear the gradient list corresponding to the current prompt variable, and output the updated prompt variable.

6. The method for dynamic experience refinement and self-evolution of an intelligent agent based on text gradient driving according to claim 1, characterized in that, The expression for the loss variable is: , in, Represents the loss variable. Let be the average accuracy of the most recent N tasks.

7. The method for dynamic experience refinement and self-evolution of an agent based on text gradient-driven approach according to claim 1, characterized in that, The structured experience unit includes the following fields: experience identifier, experience type, associated topic, applicable conditions, content text, message time, citation identifier, creation time, modification time, source author, related rating, vector representation, and extended data.

8. The method for dynamic experience refinement and self-evolution of an intelligent agent based on text gradient driving according to claim 1, characterized in that, The text gradient variable includes five attributes: text value, role description, differentiability flag, gradient list, and predecessor list.

9. A text gradient-driven intelligent agent dynamic experience refinement and self-evolution system, characterized in that, include: The first processing module is used to initialize the agent system, including: constructing the structured experience units of the agent system, initializing basic cue variables and text gradient optimizer; The second processing module is used to abstract the experience processing process of the intelligent agent system into standardized operators, and construct an experience refinement pipeline including query construction operators, experience retrieval operators, reordering operators and experience rewriting operators; and process the current task context through the experience refinement pipeline to obtain experience text adapted to the current task context. The third processing module is used to execute the current task based on the experience text adapted to the current task context, generate the execution trajectory, score the task output after execution, and generate an evaluation record. The fourth processing module is used to map the initialized basic cue variables to cue variables in the text gradient variables, combine the response variables and loss variables to form text gradient variables, construct a chained text gradient computation graph based on the text gradient variables, generate a gradient list of text gradients stored in the current node, update the cue variables according to the text gradients, output the updated cue variables, and write the updated cue variables back to the cue word template corresponding to the basic cue variables, so that they take effect when the next round of experience processing is initialized. The fifth processing module is used to calculate the utility value of experience and eliminate low-value experience based on the size of the utility value. The sixth processing module is used to repeatedly execute the second to fifth processing modules, so that the agent can continuously evolve as the task is executed without human intervention until the convergence condition is met.

10. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the text gradient-driven agent dynamic experience refinement and self-evolution method as described in any one of claims 1 to 8.

Images