Experienced iterative refinement method, device and equipment for large language model

Abstract

This invention relates to the field of natural language processing and discloses a method, apparatus, and device for iterative refinement of large language models. The method includes: extracting experience from the execution of each batch of tasks; determining a paradigm for transferring experience between different batches of tasks based on the experience; forming an experience pool based on the extracted experience; deleting experiences in the experience pool with a comprehensive score lower than a preset threshold; and retaining the remaining experience in the experience pool as high-quality experience. This invention solves the problems of poor adaptability, low generation efficiency, and low accuracy in existing large model generation software for unseen tasks.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to an empirical iterative refinement method, apparatus, and device for large language models. Background Technology

[0002] In the ongoing development of artificial intelligence, Large Language Models (LLMs) have brought transformative impacts to numerous fields, demonstrating the immense potential of human-like intelligence. While their capabilities are impressive, these models exhibit certain limitations in handling complex situations beyond simple conversation. Driven by the continuous advancements in LLMs, autonomous agents based on LLMs, possessing contextual memory, multi-step planning capabilities, and the ability to utilize external tools, have emerged, significantly enhancing the capabilities of LLMs. Furthermore, another major advancement is the integration of cooperation among multiple agents. By decomposing tasks into multiple sub-tasks and engaging in multi-turn dialogues between agents, a coherent and automated solution to the task is collaboratively provided, achieving a significant increase in autonomy and greatly reducing reliance on human intervention. The continuous improvement of LLMs' capabilities expands their ability to effectively manage a wider range of complex tasks, including mathematical reasoning, software development, game development, social simulation, and scientific research. Research in software development, a representative scenario, is emerging due to its complexity requiring a combination of natural language and programming language skills, the need for sustained effort often necessitating a deep understanding and continuous modification of the code, and the need for quantifiable metrics for code clarity.

[0003] With the development of large models and agents, a recent major breakthrough has been enabling agents to accumulate experience from previous tasks. Representative methods extract and memorize experience by replaying historical tasks, thereby efficiently solving tasks in unseen situations by retrieving experienced samples. By accumulating experience, these agents effectively avoid repeating errors and unnecessary trial-and-error processes, reducing the need for additional human intervention.

[0004] However, existing technologies often rely on heuristic rules that are generated one-time. This approach limits the ability of agents to dynamically adapt to complex tasks such as software development because it lacks the dynamic update mechanism needed to continuously adapt to new tasks. Summary of the Invention

[0005] This invention provides an empirical iterative refinement method, apparatus, and device for large language models, which solves the problems of poor adaptability, low generation efficiency, and low accuracy of existing large model generation software in unseen tasks.

[0006] An empirical iterative refinement method for large language models includes:

[0007] Learn from the experience gained during the execution of each batch of tasks;

[0008] Based on the experience described, determine the paradigm for transferring experience between different batches of tasks;

[0009] An experience pool is formed based on the extracted experience. Experiences with a comprehensive score lower than a preset threshold in the experience pool are deleted, and the remaining experience in the experience pool is retained as high-quality experience.

[0010] In one embodiment of the present invention, during the execution of each batch of tasks, an instruction sequence is generated by instructing an agent, and a solution sequence is generated by responding to an agent; wherein each solution in the solution sequence has a corresponding node; a directed task execution chain is determined according to the instruction sequence and the solution sequence; and non-adjacent node pairs in the task execution chain are traversed to obtain the experience between all non-adjacent node pairs as shortcut experience.

[0011] In one embodiment of the present invention, during the execution of each batch of tasks, the method further includes: in each interaction between the instructing agent and the responding agent, when the instructing agent receives a current solution, treating the current solution as a query query whose semantic matching degree with the current solution exceeds a first preset threshold; performing contextual reasoning on the past instructions as few-sample examples to generate an experience-enhanced new instruction and transmitting it to the responding agent; when the responding agent receives the new instruction, treating the new instruction as a query query query whose semantic matching degree with the new instruction exceeds a second preset threshold; performing contextual reasoning on the past instructions as few-sample examples to generate an experience-enhanced new solution; and treating the new instruction and new solution generated in each interaction as a tuple for use by unseen tasks.

[0012] In one embodiment of the present invention, the paradigms for transferring experience between different batches of tasks include a continuous mode and a cumulative mode.

[0013] In one embodiment of the present invention, the continuous mode is as follows: when the agent executes the current task flow, it acquires and uses the previous experience generated when the previous task flow was executed to generate the experience corresponding to the current task flow; when the agent executes the next task flow, it acquires and uses the experience generated when the current task flow was executed as the previous experience to generate the experience corresponding to the next task flow.

[0014] In one embodiment of the present invention, the accumulation mode is as follows: when the agent is executing the current task flow, it obtains the experience generated when executing all previous task flows in the experience pool as past experience and uses it to generate experience corresponding to the current task flow.

[0015] In one embodiment of the present invention, the step of forming an experience pool based on extracted experience and deleting experiences in the experience pool whose comprehensive score is lower than a preset threshold specifically includes: identifying and retaining shortcut experiences in the experience pool that show an information gain not lower than a third preset threshold during software optimization, and deleting shortcut experiences in the experience pool that show an information gain lower than the third preset threshold; retaining experiences in the experience pool whose usage frequency is not lower than a fourth preset threshold, and deleting experiences that show a usage frequency lower than the fourth preset threshold.

[0016] An empirical iterative refinement apparatus for a large language model includes:

[0017] The experience acquisition and utilization module is used to extract experience from the execution of each batch of tasks.

[0018] An experience iteration module is used to determine a paradigm for transferring experience between different batches of tasks based on the experience.

[0019] The experience filtering module is used to form an experience pool based on the extracted experience, delete the experience in the experience pool whose comprehensive score is lower than a preset threshold, and retain the remaining experience in the experience pool as high-quality experience.

[0020] An empirical iterative refining apparatus for a large language model includes:

[0021] At least one processor; and,

[0022] The memory is communicatively connected to the at least one processor via a bus; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor to implement the method as described in any of the above embodiments.

[0024] A non-volatile storage medium storing computer-executable instructions, which are executed by a processor to implement the method as described in any of the above embodiments.

[0025] This invention provides an empirical iterative refinement method, apparatus, and device for large language models, which has at least the following beneficial effects:

[0026] 1. Enhanced Learning Efficiency: Through experience iteration and filtering, large models can learn and adapt to new tasks more quickly during software development. This approach reduces the complexity of rules and computational burden that may arise during cross-task experience propagation, effectively helping multi-agent systems dynamically adapt to task flows for efficient swarm intelligence reasoning, thereby improving overall learning efficiency.

[0027] 2. Experience Accumulation and Application: Large models can not only learn from current tasks, but also utilize previous experience to solve new problems. This accumulation and application of experience makes large models more efficient and accurate when facing new challenges.

[0028] 3. Error reduction and performance improvement: As experience in the experience pool is continuously filtered and iterated, the vast majority of low-quality experience is eliminated, reducing the likelihood of using low-quality experience.

[0029] 4. Adaptability and flexibility: Compared to generating all static experience at once, the large model has greater adaptability and flexibility by continuously iterating the experience pool and dynamically updating the experience to adapt to the ever-changing unseen tasks.

[0030] 5. Improve the utilization of experience space: By filtering the experience information gain and the frequency of use, the demand for experience space in large models is significantly reduced, thus improving the utilization of experience space. Attached Figure Description

[0031] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this invention, illustrate exemplary embodiments of the invention and are used to explain the invention, but do not constitute an undue limitation of the invention. In the drawings:

[0032] Figure 1 This is a schematic diagram illustrating the steps of an empirical iterative refinement method for a large language model provided in an embodiment of the present invention;

[0033] Figure 2 This is a schematic diagram of a task execution chain constructed for gaining experience, provided in an embodiment of the present invention.

[0034] Figure 3 This is a schematic diagram illustrating the iterative refinement process of the experience pool on the task flow provided in an embodiment of the present invention.

[0035] Figure 4 A schematic diagram of an empirical iterative refinement device for a large language model provided in an embodiment of the present invention;

[0036] Figure 5 This is a schematic diagram of an empirical iterative refinement device for a large language model provided in an embodiment of the present invention. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be described clearly and completely below in conjunction with specific embodiments. Obviously, the described embodiments are only a part of the embodiments of this invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0038] It should be noted that those skilled in the art will understand, explicitly and implicitly, that the embodiments described in this invention can be combined with other embodiments without conflict. Unless otherwise defined, the technical or scientific terms used in this invention should be understood in their ordinary sense by those skilled in the art. The terms "a," "an," "an," "the," etc., used in this invention do not indicate quantity limitation and can represent singular or plural. The terms "comprising," "including," "having," and any variations thereof used in this invention are intended to cover non-exclusive inclusion; the terms "first," "second," "third," etc., used in this invention are merely to distinguish similar objects and do not represent a specific ordering of objects.

[0039] In existing technologies, the adaptability of large models to unseen software-generated tasks needs improvement. They cannot iteratively update their accumulated experience based on new tasks because the experience in existing technologies is often generated once using heuristic rules, and this static experience lacks the ability to be dynamically updated during future task execution. To address this issue, this invention proposes a collaborative decision-making method enhanced by experience iteration, comprising three core modules: an experience acquisition and utilization module, an experience iteration module, and an experience filtering module.

[0040] The following is a detailed explanation.

[0041] Figure 1 This is a schematic diagram illustrating the steps of an empirical iterative refinement method for a large language model provided in an embodiment of the present invention. Figure 1 The empirical iterative refinement method for the large language model shown can be executed by an empirical iterative refinement device for the large language model. This device can be located on a server, such as a physical server containing independent hosts, a virtual server hosted in a host cluster, or a cloud server. This embodiment of the invention does not limit the specific implementation of this method. The empirical iterative refinement method for the large language model may include the following steps:

[0042] S110: Extract experience from the execution of each batch of tasks.

[0043] In one embodiment of the present invention, during the execution of each batch of tasks, an instruction sequence is generated by instructing an agent, and a solution sequence is generated by responding to an agent; wherein each solution in the solution sequence has a corresponding node; a directed task execution chain is determined based on the instruction sequence and the solution sequence; non-adjacent node pairs in the task execution chain are traversed, and the experience between all non-adjacent node pairs is obtained as shortcut experience.

[0044] In one embodiment of the present invention, during the execution of each batch of tasks, in each interaction between the instructing agent and the responding agent, when the instructing agent receives a current solution, it treats the current solution as a past instruction whose semantic matching degree with the query exceeds a first preset threshold; it uses the past instruction as a few-sample example for contextual reasoning to generate an experience-enhanced new instruction and passes it to the responding agent; when the responding agent receives the new instruction, it treats the new instruction as a past solution whose semantic matching degree with the query exceeds a second preset threshold; it uses the past solution as a few-sample example for contextual reasoning to generate an experience-enhanced new solution; and it uses the new instruction and new solution generated in each interaction as a tuple for use by unseen tasks.

[0045] Specifically, the steps in this embodiment can be executed by the experience acquisition and utilization module, which aims to extract experience from each batch of tasks to improve the agent's ability and efficiency in solving unseen tasks.

[0046] In each batch of task sets In each task The execution of each action drives the interaction between the instruction agent and the response agent. In this process, the instruction agent provides a sequence of instructions I = {i1, i2, ..., i...}. n The responding agent responds with a sequence of solutions S = {s1, s2, ..., s}. n This dynamic interaction process can be naturally modeled as a directed task execution chain G = (N, E):

[0047] N={s j |s j ∈S}∪{s0},E={(s j i j+1 ,s j+1 )|s j ,s j+1 ∈S,i j+1 ∈I}

[0048] Where N represents the set of nodes corresponding to the solution (s0 represents the initial empty solution), and E represents the set of edges corresponding to the instructions. Each edge (s j i j+1 ,s j+1 ) describes the process from instruction i j+1 A solution for driving j To the optimized solution j+1 The transfer.

[0049] Considering that not every round of software optimization during execution is of high quality, the design obtains all "shortcut" experiences by traversing non-adjacent node pairs in the execution chain:

[0050]

[0051] in Indicates from s i to s j It is reachable by a graph path. This indicates a pseudo-instruction generated through self-guidance.

[0052] like Figure 2 The diagram shows a task execution chain constructed by the present invention to acquire experience. The task execution chain generates trajectories for various task creation software and obtains "shortcut" edges connecting non-adjacent nodes as the experience of the agent.

[0053] The task reasoning process begins with an instruction agent I equipped with "solution-to-instruction" memory, when it receives a current solution s. j First, the agent autonomously uses a retrieval tool to query past instructions with high semantic matching based on the query scheme. The retrieved instructions are then used as few-shot examples for in-context reasoning to guide the agent in generating a new, experience-enhanced instruction. Then, the responding agent A, equipped with "instruction-to-solution" memory, queries past solutions with high semantic matching based on the new instruction received (which is a query). These solutions constitute a few-sample example for the responding agent, guiding its reasoning to form a new solution. This experience-enhancing interaction process can be formalized as:

[0054]

[0055] in Let q represent the query to retrieve the top k semantic matches in the key-value database s, and I(·,e) and A(·,e) represent the instructing and responding agents' contextual reasoning operations using e as a sample, respectively. In each round of interaction, the experience-enhanced solution, in turn, serves as the context for the next interaction; therefore, the experience-enhanced task execution process can be represented as a series of tuples:

[0056]

[0057] Each tuple contains an experience-enhancing instruction and a corresponding solution that can be retrieved and utilized in unseen tasks.

[0058] S120: Determine a paradigm for transferring experience between different batches of tasks based on experience.

[0059] In one embodiment of the present invention, the paradigms for transferring experience between different batches of tasks include a continuous mode and a cumulative mode.

[0060] In one embodiment of the present invention, the continuous mode is as follows: when the agent executes the current task flow, it acquires and uses the previous experience generated when the previous task flow was executed to generate the experience corresponding to the current task flow; when the agent executes the next task flow, it acquires and uses the experience generated when the current task flow was executed as the previous experience to generate the experience corresponding to the next task flow.

[0061] In one embodiment of the present invention, the accumulation mode is as follows: when the agent is executing the current task flow, it obtains the experience generated when executing all previous task flows in the experience pool as past experience and uses it to generate experience corresponding to the current task flow.

[0062] Specifically, the steps in this embodiment can be executed through the experience iteration module, which aims to design a paradigm for transferring experience between different batch task sets.

[0063] For each batch of tasks, they should be able to utilize the experience generated by previous batches, and naturally, they can also generate new experience to pass on to the next batch. Based on this, this invention proposes two experience iteration modes: a continuous mode and a cumulative mode.

[0064] Inspired by the insight that recently acquired experience naturally aligns with the current objective of this invention, we introduce the continuous mode. This mode is used when executing task flows. At that time, the agent can be in the nearest predecessor task flow. Perform tasks and accumulate experience ε i-1 This also forms their role in the next task flow. Based on past experience. This process can be formally represented as:

[0065]

[0066] in Indicates in the task flow The above describes the use of the experience pool ε.

[0067] The cumulative model explores whether continuous experience accumulation can improve task-solving capabilities. The agent executes task flows... At that time, all previous experience pools {ε1, ε2, ..., ε} can be used. i-1 This pattern can be formalized as:

[0068]

[0069] The two experience iteration modes can be likened to the intergenerational transmission of knowledge. The continuous mode is similar to descendants inheriting knowledge from their parents, while the cumulative mode is similar to descendants inheriting knowledge from their parents and all their predecessors.

[0070] like Figure 3 This invention describes the iterative refinement process of the experience pool on the task flow. In each batch of tasks, each agent generates its new experience pool and propagates it to subsequent task flows, allowing the agent's experience to be used in unseen task sets, thereby generating a new generation of experience to empower the next batch of task sets, achieving a dual cycle of continuous accumulation and utilization of experience.

[0071] By using the two methods of dynamic iterative enhancement of experience described above, the drawback of static experience lacking the ability to be dynamically updated during future task execution is avoided, thus achieving a dual cycle of continuous accumulation and utilization of experience.

[0072] S130: Based on the extracted experience, an experience pool is formed. Experiences with a comprehensive score lower than a preset threshold in the experience pool are deleted, and the remaining experience in the experience pool is retained as high-quality experience.

[0073] In one embodiment of the present invention, an experience pool is formed based on the extracted experience, and experiences in the experience pool with a comprehensive score lower than a preset threshold are deleted. Specifically, this includes: identifying and retaining shortcut experiences in the experience pool that show an information gain not lower than a third preset threshold during the software optimization process, and deleting shortcut experiences in the experience pool that show an information gain lower than the third preset threshold; retaining experiences in the experience pool that are used at a frequency not lower than a fourth preset threshold, and deleting experiences that are lower than the fourth preset threshold.

[0074] Specifically, the steps in this embodiment can be executed by the experience filtering module. The experience filtering module is designed to take into account that the experience space may expand unexpectedly during the accumulation of experience, including a large amount of low-quality or rarely used experience.

[0075] To address this issue, this invention proposes a heuristic experience filtering mechanism based on information density and experience usage frequency. This mechanism prioritizes frequently used experiences during task execution while discarding identified low-quality experiences, thereby simplifying the evolution of the experience pool towards higher efficiency.

[0076] Selectively identify shortcut experiences in software optimization that exceed a defined threshold ∈ information gain when connecting non-adjacent nodes.

[0077]

[0078] Where sim(·,·) represents the similarity between the solution computed using cosine similarity and external embeddings and the requirements of another node or task. This represents a binary signal indicating whether compilation by an external compiler was successful.

[0079] Furthermore, a long-tailed distribution was observed in the dynamic utilization of the experience pool, implying that the tail portion is actually rarely used. By leveraging the frequency distribution of experience pool usage for each batch of tasks, certain experiences can be selectively eliminated to obtain a subset with relatively high retrieval probabilities.

[0080]

[0081] Where e represents experience, f(e) represents the retrieval frequency of e, rank(·) represents the retrieval frequency index sorted in descending order, and θ represents the quantile threshold. By combining these criteria that simultaneously consider static information gain and dynamic usage frequency, the two are combined to obtain high-quality experience:

[0082]

[0083]

[0084] Where i represents the order of the task flow.

[0085] By filtering through two layers of experience information gain and usage frequency, the demand for experience space in large models is significantly reduced, the complex rules and computational burden that may occur when propagating experience across tasks are reduced, and multi-agent systems are effectively helped to dynamically adapt to task flow for efficient swarm intelligence reasoning, thereby improving overall learning efficiency.

[0086] By implementing the above-described empirical iterative refinement scheme, the present invention can produce the following effects:

[0087] 1. Enhanced Learning Efficiency: Through experience iteration and experience filtering modules, large models can learn and adapt to new tasks more quickly during software development. This approach reduces the complexity of rules and computational burden that may arise during cross-task experience propagation, effectively helping multi-agent systems dynamically adapt to task flows for efficient swarm intelligence reasoning, thereby improving overall learning efficiency.

[0088] 2. Experience Accumulation and Application: Large models can not only learn from current tasks, but also utilize previous experience to solve new problems. This accumulation and application of experience makes large models more efficient and accurate when facing new challenges.

[0089] 3. Error reduction and performance improvement: As experience in the experience pool is continuously filtered and iterated, the vast majority of low-quality experience is eliminated, reducing the likelihood of using low-quality experience.

[0090] 4. Adaptability and flexibility: Compared to generating all static experiences at once, this method continuously iterates the experience pool and dynamically updates the experiences to adapt to constantly changing unseen tasks, thus exhibiting higher adaptability and flexibility.

[0091] 5. Improve the utilization of experience space: By filtering the experience information gain and the frequency of use, the demand for experience space in large models is significantly reduced, thus improving the utilization of experience space.

[0092] With these advantages, the empirical iterative refinement technology provided by this invention offers significant benefits in the practical application of large models, especially in software development scenarios that require rapid adaptation and resolution of complex tasks.

[0093] The above describes an empirical iterative refinement method for a large language model provided by an embodiment of the present invention. Based on the same inventive concept, the present invention also provides a corresponding empirical iterative refinement device for a large language model, such as... Figure 4 As shown, it includes:

[0094] The experience acquisition and utilization module 410 is used to extract experience from the execution process of each batch of tasks.

[0095] The experience iteration module 420 is used to determine the paradigm for transferring experience between different batches of tasks based on the experience.

[0096] The experience filtering module 430 is used to form an experience pool based on the extracted experience, delete the experience in the experience pool whose comprehensive score is lower than a preset threshold, and retain the remaining experience in the experience pool as high-quality experience.

[0097] This invention also provides a corresponding empirical iterative refinement device for large language models, such as... Figure 5 As shown, it includes:

[0098] The system includes at least one processor 502, a communication interface 504, a memory 506, and a communication bus 508; wherein the processor 502, the communication interface 504, and the memory 506 communicate with each other through the communication bus 508; the processor 502 can call the logical instructions stored in the memory 506 to cause at least one processor 502 to execute the steps of the above embodiments.

[0099] Based on the same idea, some embodiments of the present invention also provide media corresponding to the above methods.

[0100] Some embodiments of the present invention provide a storage medium storing computer-executable instructions, which are executed by a processor to implement the steps of the embodiments described above.

[0101] On the other hand, embodiments of this application also provide a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can perform the steps of the empirical iterative refinement method for a large language model provided in the above embodiments.

[0102] The various embodiments in this invention are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the device, medium, and program product embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0103] The devices, media, and program products and methods provided in the embodiments of the present invention are one-to-one correspondences. Therefore, the devices, media, and program products also have similar beneficial technical effects as their corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices, media, and program products will not be repeated here.

[0104] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process method or product that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process method or product. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process method or product that includes that element.

[0105] The above are merely embodiments of the present invention and are not intended to limit the invention. Although the present invention has been described in detail above with general descriptions and specific embodiments, modifications or improvements can be made to it, which will be obvious to those skilled in the art. Therefore, all such modifications or improvements made without departing from the spirit of the present invention fall within the scope of protection claimed by the present invention.

Claims

1. An empirical iterative refinement method for large language models, characterized in that, include: Learn from the experience gained during the execution of each batch of tasks; Based on the experience, a paradigm for transferring experience between different batches of tasks is determined, which includes a continuous mode and a cumulative mode. An experience pool is formed based on the extracted experience. Experiences with a comprehensive score lower than a preset threshold in the experience pool are deleted, and the remaining experience in the experience pool is retained as high-quality experience. During the execution of each batch of tasks, an instruction sequence is generated by instructing the agent, and a solution sequence is generated by responding to the agent; each solution in the solution sequence has a corresponding node; a directed task execution chain is determined based on the instruction sequence and the solution sequence; non-adjacent node pairs in the task execution chain are traversed, and the experience between all non-adjacent node pairs is obtained as shortcut experience; The step of forming an experience pool based on extracted experience and deleting experiences in the experience pool whose comprehensive score is lower than a preset threshold specifically includes: identifying and retaining shortcut experiences in the experience pool that show an information gain of not less than a third preset threshold during software optimization, and deleting shortcut experiences in the experience pool that show an information gain of not less than the third preset threshold; retaining experiences in the experience pool whose usage frequency is not less than a fourth preset threshold, and deleting experiences that show a usage frequency of not less than the fourth preset threshold.

2. The method of claim 1, wherein the method further comprises: During the execution of each batch of tasks, the method further includes: In each interaction between the instructing agent and the responding agent... When the instruction agent receives a current solution, it treats the current solution as a query query whose semantic matching degree with the current solution exceeds a first preset threshold; it uses the past instructions as few-shot examples for contextual reasoning to generate a new experience-enhanced instruction and passes it to the responding agent. When the responding agent receives the new instruction, it treats the new instruction as a query query whose semantic matching degree with the new instruction exceeds a second preset threshold; it then uses the past solutions as few-shot examples for contextual reasoning to generate a new experience-enhanced solution. The new instructions and solutions generated in each interaction are treated as a tuple for use by unseen tasks.

3. The method of claim 1, wherein the method further comprises: The continuous mode is as follows: when the agent executes the current task flow, it acquires and uses the previous experience generated when executing the task in the previous task flow to generate the experience corresponding to the current task flow. When the agent executes the next task flow, it retrieves the experience generated during the execution of the current task flow as past experience and uses it to generate experience corresponding to the next task flow.

4. The method of claim 1, wherein the method further comprises: The accumulation mode is as follows: When the agent is executing the current task flow, it retrieves the experience generated from all previous task flows in the experience pool as past experience and uses it to generate experience corresponding to the current task flow.

5. An empirical iterative refinement device for a large language model, characterized in that, include: The experience acquisition and utilization module is used to extract experience from the execution of each batch of tasks. During the execution of each batch of tasks, an instruction sequence is generated by instructing the agent, and a solution sequence is generated by responding to the agent; each solution in the solution sequence has a corresponding node; a directed task execution chain is determined based on the instruction sequence and the solution sequence; non-adjacent node pairs in the task execution chain are traversed, and the experience between all non-adjacent node pairs is obtained as shortcut experience; An experience iteration module is used to determine a paradigm for transferring experience between different batches of tasks based on the experience. The paradigm for transferring experience between different batches of tasks includes a continuous mode and a cumulative mode. The experience filtering module is used to form an experience pool based on the extracted experience, delete the experience in the experience pool whose comprehensive score is lower than a preset threshold, and retain the remaining experience in the experience pool as high-quality experience. The step of forming an experience pool based on extracted experience and deleting experiences in the experience pool whose comprehensive score is lower than a preset threshold specifically includes: identifying and retaining shortcut experiences in the experience pool that show an information gain of not less than a third preset threshold during software optimization, and deleting shortcut experiences in the experience pool that show an information gain of not less than the third preset threshold; retaining experiences in the experience pool whose usage frequency is not less than a fourth preset threshold, and deleting experiences that show a usage frequency of not less than the fourth preset threshold.

6. An empirical iterative refinement device for a large language model, characterized in that, include: At least one processor; as well as, The memory is communicatively connected to the at least one processor via a bus; wherein, The memory stores instructions executable by the at least one processor, which are executed to implement the method as described in any one of claims 1-4.

7. A non-volatile storage medium storing computer-executable instructions, characterized in that, The computer-executable instructions are executed by a processor to implement the method as described in any one of claims 1-4.

Images