Self-evolution agent program memory system and agent

Abstract

The application discloses a self-evolutional agent program memory system, which comprises a program memory content module, an internal logic fragment code, an internal function function block and an external tool; a program memory generation update module, which is used for generating or updating program memory content; a program memory retrieval module, which is used for retrieving and recalling relevant program memory tools according to different task types and scenes; and a program memory execution module, which is used for executing the program memory tools retrieved by the program memory retrieval module and performing post-processing. The application can solve the problem that the existing agent memory system lacks a program memory component system, thereby causing limitations in flexibility, execution efficiency and adaptability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and particularly relates to a self-evolving Agent program memory system and an intelligent agent. BACKGROUND

[0002] With the development of large model technology, a series of applications based on large models have emerged. Agent (Intelligent Agent) has become one of the core applications of large model technology. Agent is a computing entity with autonomous decision-making ability, which can achieve specific goals or tasks through behaviors such as perceiving the environment, processing information and responding to behaviors, and can be widely applied in various fields such as robots, intelligent assistants, game AI, autonomous vehicles, etc. The memory system module of Agent is one of its core components, and its memory module usually simulates the human memory system to enhance the natural interaction and decision-making ability of Agent. According to the difference of function and persistence, the existing Agent memory is mainly divided into two categories: long-term memory and short-term memory.

[0003] Long-term memory (Long-term Memory, LTM) refers to the memory form that stores a large amount of knowledge and past experience, and its main feature is that the information is stored for a long time, even throughout the life cycle of the Agent. The data in long-term memory is usually used to affect the long-term decision and behavior pattern of the Agent, such as learning task rules, environment feature recognition, etc. In practical applications, long-term memory can be trained and optimized through various machine learning algorithms, such as deep learning networks or reinforcement learning strategies, so as to adapt to complex and variable task requirements.

[0004] Short-term memory (Short-term Memory, STM) focuses on information related to current activities, and usually only retains data for a short time. Short-term memory enables the Agent to quickly respond to environmental changes and temporary tasks, and handles immediate computing and decision-making needs. For example, in navigation or real-time dialogue tasks, short-term memory helps the Agent maintain awareness and response to recent events. Such memory has fast update speed, relatively small capacity, and focuses on processing and responding to new stimuli or task requirements.

[0005] If the Agent is compared to the human brain, these memories can help the Agent to think and make decisions. However, the existing memory systems rarely mention how to do it, the specific steps of execution. That is, the existing Agent memory component system ignores the importance of procedural memory, which is responsible for storing the procedures or strategies required by the Agent to perform specific tasks. This type of memory is closely related to specific operation modes and skills, such as the execution steps of a specific algorithm, the application of rules, etc. Procedural memory enables the Agent to automatically execute complex behavior sequences without significant reflection or consciousness. In the existing Agent memory system, the part of procedural memory is often not fully emphasized or developed, which raises several problems in practical applications:

[0006] 1) Insufficient flexibility in task execution: Without a dedicated procedural memory system, the Agent may not be able to quickly adapt to new or changing task execution requirements. For example, the Agent now has a tool to recite text content. Now give this tool a picture full of text content. At this time, this recitation tool is powerless. If the Agent has the ability to automatically recognize the text content of the picture, through the ability of procedural memory, the Agent will automatically convert the picture to text through OCR, and then complete the recitation task, so as to quickly adapt to the new scene.

[0007] 2) Separation of learning and execution: In the traditional model, the separation of long-term memory and short-term memory may lead to inconsistency between the learning process and the application of strategies during task execution, affecting the overall performance of the Agent. For example, the user needs to query the weather of a certain city today. Such a clear and simple requirement should be quickly responded by the Agent, which should have the ability to directly call the weather query API, rather than going through a series of task thinking, task planning, and task execution before finally giving an answer. This seriously affects the efficiency of the Agent.

[0008] 3) Limitations of memory migration and evolution: The lack of procedural memory makes it difficult for the Agent to migrate the learned strategies and skills to new situations when facing environmental changes, limiting its adaptability and continuous learning ability in different environments. SUMMARY

[0009] The purpose of the present application is to provide an Agent program memory system that can evolve itself, an intelligent agent, to solve the problem of the lack of procedural memory component system in the existing Agent memory system, thereby solving the problem of the limitations of flexibility, execution efficiency, and adaptability.

[0010] To achieve the above purpose, the following technical solutions are adopted:

[0011] An Agent program memory system that can evolve itself, comprising

[0012] program memory content module, which comprises internal logic fragment code, internal function function block and external tool;

[0013] program memory generation update module, which is used for generating or updating program memory content;

[0014] program memory retrieval module, which is used for retrieving relevant program memory tools according to different task types and scenes;

[0015] program memory execution module, which is used for executing program memory tools retrieved by the program memory retrieval module and performing post-processing.

[0016] Further, the program memory generation update module comprises a context collection module, an intention recognition module, a manual code generation module and a tool uploading module.

[0017] Further, the specific steps of the program memory generation update module for generating or updating program memory content are as follows:

[0018] S1: accept external signals;

[0019] S2: collect and load the context information of the current Agent based on the context collection module;

[0020] S3: analyze the input content of the user based on the intention recognition module to identify the user's intention;

[0021] S4: manually generate program memory content based on the manual code generation module;

[0022] S5: upload or update program memory content to the code repository based on the tool uploading module.

[0023] Further, the program memory generation update module comprises a context collection module, an intention recognition module, a code automatic generation module, a test data mock module, an automatic code debug module and a tool uploading module.

[0024] Further, the specific steps of the program memory generation update module for generating or updating program memory content are as follows:

[0025] S1: accept external signals;

[0026] S2: collect and load the context information of the current Agent based on the context collection module;

[0027] S3: analyze the input content of the user based on the intention recognition module to identify the user's intention;

[0028] S4: automatically generate program memory content based on the code automatic generation module;

[0029] S5: generating test data based on the test data mock module;

[0030] S6: executing and debugging the program memory content based on the automatic code debug module;

[0031] S7: repeating the steps S4-S7 until a qualified program memory content is generated or the number of cycles reaches a preset upper limit of cycles;

[0032] S8: uploading or updating the program memory content to the code repository based on the tool uploading module.

[0033] Further, the program memory retrieval module includes a semantic retrieval module and a tool cache module for caching common tools. The specific steps of the program memory retrieval module for retrieving and recalling relevant program memory tools are as follows:

[0034] S1: accepting an external signal;

[0035] S2: based on the tool hotword information in the external signal, triggering the tool cache module to recall relevant program memory tools, or based on the semantic retrieval module to analyze the external signal, and based on a similarity algorithm to recall a number of program memory tools related to the external signal;

[0036] S3: inputting the relevant program memory tools retrieved and recalled to the program memory execution module.

[0037] Further, the program memory retrieval module includes a subtask splitting module and a tool scheduling module. The specific steps of the program memory retrieval module for retrieving and recalling relevant program memory tools are as follows:

[0038] S1: receiving an external signal;

[0039] S2: based on the subtask splitting module, splitting the complex task input by the user in the external signal into a number of simple subtasks;

[0040] S3: based on each subtask, respectively recalling program memory tools related to it;

[0041] S4: based on the program memory tools recalled by the subtasks, adjusting the input and output formats in the tool chain based on the tool scheduling module to form a tool scheduling table;

[0042] S5: inputting the tool scheduling table to the program memory execution module.

[0043] Further, the specific steps of the program memory execution module for executing and post-processing the program memory tools retrieved by the program memory retrieval module are as follows:

[0044] S1: Based on the program memory retrieval module retrieved program memory tools and input external signal, analyze the parameters of the program memory tools, form a parameter pair of parameter-parameter value;

[0045] S2: Call the executor, feed the parameter pair of the parsed program memory tool to the tool function entity;

[0046] S3: The execution result information and error information are fed back to the other brain areas of the agent.

[0047] Also provided is an intelligent agent comprising the above system.

[0048] With the above scheme, the beneficial effects of the present application are:

[0049] 1) By constructing a self-evolving program memory system, the agent can quickly adjust its behavior and strategy according to different task requirements and environmental changes, and adapt to various complex and dynamic task environments;

[0050] 2) The system stores relevant program memory elements such as tool code and function blocks in advance, so that the agent can respond quickly when facing specific tasks without reprogramming or starting from scratch, especially in the fast reaction mode of program memory retrieval, the quick recall of commonly used tools reduces the processing time and improves the response speed and user experience;

[0051] 3) The system can realize automatic program memory generation and update, reducing the need for manual intervention, and the self-evolution ability of the agent can reduce unnecessary computing and storage resource consumption, and the process of automatically generating and optimizing tool code makes resource utilization more efficient, especially in resource-limited environments, it performs well;

[0052] 4) The system arranges complex tool chains in the slow thinking mode, so that the agent can handle more complex task scenarios such as multi-step tasks and multi-tool collaborative work, which not only enhances the problem-solving ability of the agent, but also makes the decision-making process more accurate and efficient;

[0053] 5) The system supports users to add new program memory elements according to specific needs, enhancing the scalability of the system, which provides highly customized solutions for specific industries or specific problems, meets the changing market demand, and the program memory update module contains automatic testing function, which can ensure the feasibility and functionality of the code, thereby reducing errors in actual application, in addition, through the cycle optimization iteration process, the agent can constantly self-improve, improve the overall performance and stability. BRIEF DESCRIPTION OF DRAWINGS

[0054] Figure 1A schematic block diagram of a program memory generation and update module of the present application;

[0055] Figure 2 A schematic block diagram of one embodiment of a program memory retrieval module of the present application;

[0056] Figure 3 A schematic block diagram of another embodiment of a program memory retrieval module of the present application;

[0057] Figure 4 A schematic block diagram of a program memory execution module of the present application. DETAILED DESCRIPTION

[0058] The present application will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0059] Referring to Figures 1 to 4 The present application provides a self-evolving Agent program memory system, which comprises

[0060] A program memory content module, which comprises internal logic fragment codes, internal function function blocks and external tools;

[0061] A program memory generation and update module, which is used at least for generating or updating program memory content;

[0062] A program memory retrieval module, which is used at least for retrieving and recalling relevant program memory tools according to different task types and scenarios;

[0063] A program memory execution module, which is used at least for executing program memory tools retrieved by the program memory retrieval module and performing post-processing.

[0064] Analogous to the human brain memory system, procedural memory is those memories that are implicit to human memory, mainly related to skills, which enables humans to have different skills and use different tools. For example, once a person learns to ride a bicycle, the next time he needs to ride a bicycle, he does not need to use the instruction manual or relearn it, his body has already remembered the movements and steps of riding a bicycle; the agent brain area based on a large model provides the ability of the agent to think and make decisions, and a series of tool or function logic blocks externally or internally provided to the agent give the agent execution ability; for specific tasks, the agent does not need to regenerate these codes and tools every time, but directly calls these tools, and the agent brain can not perceive the internal execution logic of these tool blocks; therefore, these tools in the agent architecture in the invention, including internal logic code fragments, internal function function blocks, external tools or API, etc. of the agent during execution are abstracted into the procedural memory of the agent; the basic elements of the procedural memory in the system are tools, and for each tool, important technical features such as tool name, tool description, tool parameter, tool body (code), tool output are included; the agent procedural memory system proposed in the invention is composed of a series of procedural memory content modules and a procedural memory management module, wherein the procedural memory management module includes a procedural memory retrieval module, a procedural memory generation update module, and a procedural memory execution module.

[0065] Based on the above, the procedural memory content in the invention includes code logic blocks, internal function functions and external tools (API) during agent execution; therefore, the generation of procedural memory is the generation of this part of code, and all codes in the system, including agent built-in code (function) and external tool function related calls, are placed in the code repository for hosting, and the processing and generation of procedural memory is to add tool code to the code repository.

[0066] In the invention, two methods of generating procedural memory are proposed, namely manually generating procedural memory and automatically generating procedural memory by the agent, wherein the manually generated procedural memory is to manually upload tool code to the code repository, and the agent implicitly forms procedural memory by loading related tool metadata, as follows:

[0067] In an embodiment, the procedural memory generation update module includes a context collection module, an intent recognition module, a manual code generation module, and a tool upload module; the procedural memory generation update module generates or updates the specific steps of the procedural memory content as follows:

[0068] S1: accept external signals;

[0069] S2: collect and load the context information of the current agent based on the context collection module;

[0070] S3: parsing the input content of the user based on the intention recognition module to identify the user's intention;

[0071] S4: manually generating program memory content based on the manual code generation module;

[0072] S5: uploading or updating the program memory content to the code repository based on the tool upload module.

[0073] In this embodiment, the program memory is generated manually, as shown on the left side of the figure Figure 1 After the Agent receives the external signal, the Agent loads the current context of the Agent, such as the resource situation, and other contents such as the tool name and description of the existing program memory, using the context collection module of the Agent. At the same time, the Agent also interprets the input content of the user to identify the user's intention. After loading the context and understanding the user's intention, the Agent can manually upload the tool memory code to the code repository.

[0074] In another embodiment, the program memory generation and update module includes a context collection module, an intention recognition module, a code automatic generation module, a test data mock module, an automatic code debug module, and a tool upload module. The specific steps of the program memory generation and update module to generate or update the program memory content are as follows:

[0075] S1: accept external signals;

[0076] S2: collect and load the current context information of the Agent based on the context collection module;

[0077] S3: parsing the input content of the user based on the intention recognition module to identify the user's intention;

[0078] S4: automatically generating program memory content based on the code automatic generation module;

[0079] S5: generating test data based on the test data mock module;

[0080] S6: executing and debugging the program memory content based on the automatic code debug module;

[0081] S7: repeating steps S4-S7 until the qualified program memory content is generated, or the number of cycles reaches the preset upper limit of the cycle;

[0082] S8: uploading or updating the program memory content to the code repository based on the tool upload module.

[0083] In this embodiment, the program memory is automatically generated, as shown in Figure 1As shown in the right side of the figure, in order to ensure the feasibility and usability of the tool, the Agent not only needs the code automatic generation module to generate the relevant tool code, but also needs to generate the mock data for testing the relevant code through the test data mock module, and execute the automatic code test through the automatic code debug module. This process may need to be repeated multiple times, and be continuously optimized and iterated until the tool function is met or the upper limit of the number of iterations is reached, which is the key to the Agent's self-evolution. Finally, the generated code is uploaded to the code repository, and the context collection module is notified to add the relevant information of the new tool, including the tool name, tool parameters, tool description, etc., so that the Agent can implicitly remember the new tool as a program. The program memory update process also follows the program memory generation process. In the present scheme, the intention recognition, automatic code generation, and automatic test data generation modules in the above are all implemented through the Prompt technology and in-context learning method.

[0084] In summary, the workflow of the program memory generation and update module can be summarized as follows:

[0085] 1.1 Accept external signals;

[0086] 1.2 Collect the current context and analyze the user's intention;

[0087] 1.3 Generate (update) program memory code;

[0088] Since the program memory generated in the present application has two generation methods, and the two methods can be parallel, there is no sequence.

[0089] Among them, the manual program memory generation:

[0090] (1) Generate program memory tool body function code;

[0091] (2) Generate program memory related metadata (tool name, tool description, tool parameters, output format, etc.).

[0092] Automatic program memory generation:

[0093] (1) Generate (adjust) the prompt related to automatic tool generation according to the user's intention and the current context;

[0094] (2) Generate the tool body and test data according to the prompt in (1);

[0095] (3) Debug / execute the tool using the test data generated in (2);

[0096] (4) Repeat the above steps until a qualified tool is generated or the number of retries reaches the upper limit of the number of iterations.

[0097] 1.4 Upload tool code and metadata: upload the manually or automatically generated code in 1.3 to the code repository and record its metadata information.

[0098] In the present application, the program memory retrieval module mainly recalls relevant program memories according to external signals, and in the present system, the name of the recall tool can be used. In order to meet different task types and scenarios, two retrieval methods of slow thinking and fast reaction are proposed in the present system. Slow thinking is mainly applied to scenarios where the process steps are complex and multiple tools need to be combined for use, such as the existing recruitment agent program memory tool library as follows: resume searching tool, chatting tool, interview invitation tool, etc. Now the user needs to find a suitable resume on the resume platform, then the candidate communicates the job content, and finally initiates an interview invitation. This scenario is relatively complex and requires multiple tools to work together. Therefore, the agent needs to think about the execution of each tool and the input-output relationship between tools, which is the slow thinking program memory retrieval method in the present system. Fast reaction program memory retrieval is mainly aimed at simple tasks, such as querying the weather and time, etc. These simple tasks require the agent to quickly respond and shorten the response time to improve user experience.

[0099] In an embodiment, the program memory retrieval module includes a semantic retrieval module and a tool cache module for caching commonly used tools. The specific steps of the program memory retrieval module for recalling relevant program memory tools are as follows:

[0100] S1: accept external signals;

[0101] S2: based on the tool hotword information in the external signal, trigger the tool cache module to recall relevant program memory tools, or based on the semantic retrieval module to analyze the external signal, and based on the similarity algorithm to recall a number of program memory tools related to the external signal;

[0102] S3: input the relevant program memory tools retrieved to the program memory execution module.

[0103] This embodiment is the fast reaction retrieval method described above, which is mainly achieved through the semantic retrieval module and the tool cache module. The tool cache module is mainly used for the quick recall of commonly used basic tools. For example, a tourism agent has functions such as ticket purchase and strategy making, and simple functions such as weather checking and geographic location obtaining are almost used. Therefore, these popular basic tools can be pre-cached, and then based on the tool hotword information in the external input signal, the tool cache module is triggered to recall relevant program memory tools. The semantic retrieval module returns the most relevant tool names for the user input signal, and finally inputs the retrieved tool metadata to the tool execution area. The specific workflow can be referred to as shown in Figure 2 ​

[0104] In an embodiment, the program memory retrieval module includes a subtask splitting module and a tool scheduling module. The specific steps of the program memory retrieval module for retrieving relevant program memory tools are as follows:

[0105] S1: receiving an external signal;

[0106] S2: based on the subtask splitting module, the complex task input by the user in the external signal is split into several simple subtasks;

[0107] S3: based on each subtask, relevant program memory tools are recalled respectively;

[0108] S4: based on the tool scheduling module, the input and output formats in the tool chain are adjusted according to the program memory tools recalled by the subtasks, and a tool scheduling table is formed;

[0109] S5: the tool scheduling table is input to the program memory execution module.

[0110] This embodiment is the slow thinking retrieval method described above, which is mainly reflected in the arrangement of complex tool chains. First, the subtask splitting module of the tool arrangement center splits the user input information into one or more subtasks. Then, each subtask can recall relevant program memory metadata according to its own needs. Then, the split subtasks and the metadata of each subtask are returned to the tool arrangement center. The tool scheduling module of the tool arrangement center adjusts the tool input and output forms according to the input and output between tasks, and then forms a complete tool scheduling table. This process may need to be repeated several times to perfect it. From the implementation point of view, it is a large Prompt containing tool details and input and output conversion between related tools. This tool scheduling table is the program memory of this complex scene, which can be used as a temporary task or can be solidified into a new complex program memory. The specific workflow can be referred to in Figure 3 .

[0111] In an embodiment, the program memory execution module executes the program memory tools retrieved by the program memory retrieval module and performs post-processing. The specific steps are as follows:

[0112] S1: based on the program memory tools retrieved by the program memory retrieval module and the input external signal, the parameters of the program memory tools are parsed to form parameter pairs of parameters-parameter values;

[0113] S2: calling an executor, feeding the parsed parameter pairs of the program memory tools to the tool function entity;

[0114] S3: feeding the execution result information and error information to the Agent other brain area.

[0115] In this embodiment, the program memory execution module mainly performs execution and post-processing operations based on the retrieved tools; during operation, it refers to... Figure 4 As shown, firstly, standardized tool parameters are parsed from external signals and tool metadata; then, the executor is called to execute the code; finally, the results are post-processed. The tool parameter generation module needs to combine tool metadata information, including package name, tool description information, tool parameter name, etc., and user signals, and use Prompt to generate parameter pairs corresponding to the tool parameters' parameter names and values; after the execution tool model calls the executor to execute the program memory code, it will send feedback information, such as execution result information or anomalies, to other brain regions of the Agent for further decision-making.

[0116] In addition, an intelligent agent is also provided, which includes the system described above. Its working principle is the same as described above and will not be repeated here.

[0117] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A self-evolving agent program memory system, characterized in that, include The program memory content module includes internal logic fragment code, internal function blocks, and external tools. The program memory generation and update module is used at least to generate or update program memory content; The program memory retrieval module is used at least to retrieve relevant program memory tools based on different task types and scenarios; The program memory execution module is at least used to execute the program memory tools retrieved by the program memory retrieval module and perform post-processing. The program memory generation and update module includes a context collection module, an intent recognition module, a manual code generation module, and a tool upload module; The specific steps for the program memory generation and update module to generate or update the program memory content are as follows: S1: Receive external signals; S2: The context collection module collects and loads the context information of the current Agent; S3: Based on the intent recognition module, the user's input is parsed to identify the user's intent; S4: Manually generate program memory content based on the manual code generation module; S5: Upload or update program memory content to the code repository based on the tool upload module; The program memory generation and update module includes a context collection module, an intent recognition module, an automatic code generation module, a test data mock module, an automatic code debugging module, and a tool upload module; The specific steps for the program memory generation and update module to generate or update the program memory content are as follows: S1: Receive external signals; S2: The context collection module collects and loads the context information of the current Agent; S3: Based on the intent recognition module, the user's input is parsed to identify the user's intent; S4: Automatically generates program memory content based on the code generation module; S5: Generates test data based on the test data mock module; S6: Executes and debugs program memory based on the automatic code debug module; S7: Repeat steps S4 to S7 until qualified program memory content is generated, or the number of loops reaches the preset loop limit. S8: Upload or update program memory content to the code repository based on the tool upload module; The program memory retrieval module includes a semantic retrieval module and a tool caching module for caching commonly used tools; the specific steps for the program memory retrieval module to retrieve and recall relevant program memory tools are as follows: S1: Receive external signals; S2: Based on the tool hot word information in the external signal, trigger the tool cache module to recall the relevant program memory tools, or based on the semantic retrieval module to parse the external signal and recall several program memory tools related to the external signal based on the similarity algorithm; S3: Input the relevant program memory tools for retrieval and recall into the program memory execution module; The program memory retrieval module includes a subtask splitting module and a tool scheduling module; the specific steps for the program memory retrieval module to retrieve and recall relevant program memory tools are as follows: S1: Receive external signals; S2: Based on the subtask decomposition module, the complex task input by the user in the external signal is decomposed into several simple subtasks; S3: Based on each subtask, recall the program memory tools associated with it; S4: Based on the program memory tool recalled by the subtask, and based on the tool scheduling module, adjust the input and output formats in the toolchain to form a tool scheduling table; S5: Input the tool schedule into the program memory execution module; The specific steps of the program memory execution module executing the program memory tool retrieved by the program memory retrieval module and performing post-processing are as follows: S1: Based on the program memory tool retrieved by the program memory retrieval module and the input external signal, the parameters of the program memory tool are analyzed to form parameter pairs of parameter-parameter values; S2: Call the executor and feed the parsed program memory tool's parameter pairs to the tool function entity; S3: Feedback the execution results and error messages to other brain regions of the Agent; The basic element of the program's memory content is a tool. Each tool includes a tool name, tool description, tool parameters, tool code, and tool output. All code in this system is hosted in a code repository. When generating program memory content, program memory-related metadata is generated, and the program memory content is uploaded to or updated to the code repository through the tool upload module.

2. An intelligent agent, characterized in that, Includes the system described in claim 1.

Images