An agent long-period task planning method based on nonlinear gate and time decay

Abstract

The application discloses an agent long-period task planning method based on nonlinear gate control and time decay, and belongs to the technical field of artificial intelligence; the method constructs a three-layer storage architecture of core memory, working memory and scene memory, firstly writes user initial core constraints into the core memory and permanently resides; high-dimensional feature coding is performed on each new input, a time exponential decay function is used to dynamically calculate the retention weight of each record in the working memory, and records lower than a threshold value are archived to the scene memory; when searching the scene memory, whether the similarity exceeds a hard threshold value is judged through nonlinear semantic gating, only the qualified records are combined with time decay and inherent importance to calculate a comprehensive awakening score, and high-score records are selected; finally, the three-layer memories are spliced according to templates and input into a large model for action planning, and the execution result is written back to the working memory to form a closed loop.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and natural language processing, specifically to a long-cycle task planning method for intelligent agents based on nonlinear gating and time decay. Background Technology

[0002] In recent years, with the rapid development of Large Language Model (LLM) technology, a large number of intelligent agent applications based on large models have emerged. These intelligent agents can autonomously understand the complex intentions of users and complete complex tasks such as automatic programming, data analysis, and schedule management through multi-round tool calls and environmental interactions. Task planning is the core capability of an intelligent agent, which means accurately deciding on the next action based on user instructions, environmental feedback, historical context, and other information. In real-world applications, many tasks are long-cycle and multi-round. Historical trial and error experience and initial core constraints are of great value to subsequent decisions. Therefore, memory management and task planning capabilities directly determine the task completion rate and user experience of the intelligent agent.

[0003] Existing long-term memory management methods have the following significant drawbacks:

[0004] First, mainstream methods rely on sliding windows or simple text summarization to manage context. As the number of interaction rounds increases, sliding windows will directly truncate and lose early key constraints, causing "catastrophic forgetting"; while summarization methods will blur technical details and cannot solve the "loss of intermediate attention" problem inherent in large models, resulting in a significant drop in the accuracy of extracting key information located in the middle of the context, causing planning deviations.

[0005] Second, some methods introduce vector retrieval to recall historical memories, but only use a single cosine similarity ranking, which can only measure semantic relevance and cannot distinguish between "highly relevant but outdated and useless" memories and "highly relevant and recently critical" memories, resulting in confusion in memory recall priorities; at the same time, the indiscriminate ranking and comparison of all historical records leads to a linear increase in token consumption with running time, resulting in high inference costs.

[0006] Therefore, there is an urgent need for a new long-term task planning method that can hierarchically manage memories at different lifecycles, intelligently filter irrelevant information, prioritize the recall of key historical experiences, and control the context length while ensuring planning accuracy. Summary of the Invention

[0007] The purpose of this invention is to provide a long-cycle task planning method for intelligent agents based on nonlinear gating and time decay, which solves the problems of forgetting key information, confusing memory wake-up priorities, and excessive consumption of context tokens in existing technologies for long-cycle tasks.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: a long-cycle task planning method for intelligent agents based on nonlinear gating and time decay, comprising the following steps:

[0009] Step 1: Construct a three-layer memory architecture, namely the core memory area that stores core constraints and does not participate in decay, the working memory area that uses a queue to store recent interactions, and the context memory area that uses vectors to store historical experience. Write the user's initial global constraints into the core memory area and keep them permanently in the large model context.

[0010] Step 2: Receive new user instructions or environmental observations in each round, and use a text embedding model to encode the input with high-dimensional features to obtain the current state feature vector;

[0011] Step 3: Calculate the retention weight after time decay for each historical record in the working memory area, and remove records with retention weights lower than the archiving threshold from the working memory area and write them to the context memory area for storage.

[0012] Step 4: Retrieve the context memory area based on the current state feature vector and adopt a non-linear semantic gating mechanism: only when the cosine similarity between a record and the current state exceeds a preset hard threshold, the comprehensive wake-up score is calculated by combining the time decay factor and inherent importance; records with similarity below the threshold are directly filtered out.

[0013] Step 5: Select the episodic memory records with the highest comprehensive arousal scores, and combine the core memory, the selected episodic memory, and the current content of working memory according to a fixed template. Input the data into the large language model to deduce the current round of execution actions.

[0014] Step 6: Execute the action in the environment, obtain the execution result, write the action and result as the latest record into the working memory area, increment the round by 1 and return to step 2 until the task is completed.

[0015] In a preferred embodiment of the present invention, step 1 specifically includes:

[0016] Step 1.1: The core memory area stores the user's initial global constraints, safety criteria, and core objectives, assigns the highest scheduling weight, does not participate in any decay calculations, and permanently resides in the input context of the large model;

[0017] Step 1.2: The working memory area adopts a first-in-first-out queue structure to store the interaction logs of the most recent rounds, the current subtask status, and short-cycle environmental feedback;

[0018] Step 1.3: The context memory area is implemented using a vector database built on the approximate nearest neighbor algorithm. It persistently stores historical interaction records and trial-and-error experiences archived from the working memory area. Each record contains four fields: text content, feature vector, generation timestamp, and inherent importance.

[0019] In a preferred embodiment of the present invention, in step 2, a deep learning text embedding model is used to embed the input text. Perform high-dimensional space mapping to obtain the feature vector of the current state. :

[0020]

[0021] Among them, feature vector This represents the coordinates of the current task situation in the multidimensional semantic space, which will be used for subsequent intent matching.

[0022] In a preferred embodiment of the present invention, in step 3, the first [item] in the working memory 1 record, which is in the current Retention weight at any time The calculation formula is:

[0023]

[0024] in, Assign an initial base weight to this interaction record to the system. It is a natural constant. This indicates the time difference or round difference between the occurrence of the record and the current round. The preset decay rate hyperparameter for the system, It's not a fixed value; it's dynamically bound to the agent's current task type: when performing tasks with extremely high short-term context requirements, the value is increased. Values ​​that promote rapid forgetting, i.e., short half-life; when performing long-term observation tasks, adjust the value to achieve this. This allows for slow forgetting, i.e., a long half-life;

[0025] When the retention weight of a certain record Falling below the system's preset archiving threshold When this happens, the system removes it from the working memory area and writes its text and corresponding feature vector into the context memory area for deep hibernation storage.

[0026] In a preferred embodiment of the present invention, step 4 specifically includes:

[0027] Let the current state feature vector be The first in the episodic memory bank Historical feature vectors for A 3D floating-point array, whose generation time is... The importance of static foundations is Define the overall wake-up score for this record. The calculation formula is as follows:

[0028]

[0029]

[0030] The parameter definitions and control logic are as follows:

[0031] and Let represent the magnitudes of the two vectors respectively, and let be the cosine similarity operator, representing the similarity between the current situation and historical memory in the core semantic direction; The system has a preset core semantic hard threshold; For unit step gate function, when The output is 1 when the condition is met, and 0 otherwise. This is a regulating factor used to balance the weighting ratio between relevance and inherent importance;

[0032] Comprehensive wake-up score of all records in the system's computing database The records with the highest scores and greater than 0 are extracted from the context memory area and added to the candidate set.

[0033] As a preferred embodiment of the present invention, the cosine similarity operator constructs a nonlinear semantic gate:

[0034] Only when the cosine similarity crosses the hard threshold At that time, the gating is activated, and the subsequent time decay operator... With inherent importance operator Only then is it activated to participate in the scoring; if the similarity does not meet the standard, the overall score of the entire memory is directly reduced to zero and it is blocked by the system.

[0035] In a preferred embodiment of the present invention, in step 5, the prompt words for the final input large language model are constructed as follows:

[0036]

[0037] Here, ⨁ represents structured assembly, where the system combines absolute principles, historical experience, and the complete context of the current situation. The input is fed into a large language model, which performs logical reasoning based on the context and outputs the first... The specific action instructions to be executed for each step.

[0038] In a preferred embodiment of the present invention, in step 6, the intelligent agent executes the generated action in the external environment or software system and obtains execution feedback including API return results, system errors, and state changes.

[0039] Package the action instructions and execution results of this round into a new interaction record, assign it an initial base weight, and push it to the end of the working memory queue; Round Promote as Then return to step 2 to enter the next cycle until the overall task objective is achieved or the termination condition is triggered.

[0040] The beneficial effects of this invention are as follows:

[0041] (1) The present invention adopts a three-layer memory hierarchical architecture, and the core memory is permanently retained without decay, which avoids the catastrophic forgetting problem caused by the initial constraint conditions being truncated by the sliding window from the mechanism.

[0042] (2) This invention proposes a nonlinear semantic gating mechanism, which uses cosine similarity as a hard threshold rather than a ranking criterion, and directly filters out irrelevant memories, thus avoiding the priority confusion caused by traditional single-dimensional similarity retrieval recalling a large number of semantically similar but useless historical records.

[0043] (3) The present invention adopts time-exponential decay dynamic archiving working memory to remove low-weight records from the context of large models in a timely manner, so as to keep the token amount of a single inference stable, fundamentally alleviate the problem of attention loss in the middle of large models, and significantly reduce the inference cost.

[0044] (4) The memory scheduling of the present invention is based entirely on objective mathematical formulas and does not rely on the large model itself to determine whether it is necessary to retrieve history. This reduces memory omissions caused by model illusions and improves the robustness and stability of long-cycle task execution. Attached Figure Description

[0045] Figure 1 This is a schematic diagram of the overall process of the intelligent agent long-cycle task planning method based on nonlinear gating and time decay of the present invention. Detailed Implementation

[0046] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0047] This invention proposes a long-cycle task planning method for intelligent agents based on nonlinear gating and time decay. Addressing the problem of agent memory management and task planning in long-cycle, multi-turn interaction scenarios, it constructs a three-layer architecture of core memory, working memory, and contextual memory. Intelligent memory scheduling is achieved through nonlinear gating and time decay algorithms. The overall process is as follows: Figure 1 As shown.

[0048] Step 1: Multi-level memory initialization and core constraint locking

[0049] Before the agent initiates a long-cycle task, the system constructs three independent memory storage modules for it, each with a different lifecycle and scheduling priority:

[0050] (1) Core Memory: This area stores the user's initial global constraints, safety rules, or core objectives. For example, hard constraints that run throughout the entire task cycle, such as "the entire project uses Python 3.10 syntax," "all files are saved to the D drive work directory," and "no user data should be deleted." This area is given the highest absolute weight, does not participate in decay calculations, and permanently resides in the input context of the large model, and will not be discarded as the number of rounds increases.

[0051] (2) Working Memory: This uses a queue structure to store the interaction logs of the most recent rounds, the status of the currently executed subtasks, and short-cycle environmental feedback. For example, the content of the dialogue in the last 3-5 rounds, the return result of the tool called in the previous step, the file name currently being edited, etc., which is the main source of context for the agent's real-time inference.

[0052] (3) Episodic Memory: This is implemented using a vector database built based on an approximate nearest neighbor algorithm (such as HNSW). It is used to persistently store historical interaction records, early trial and error experiences, and summaries of completed subtasks extracted from the working memory. Each record contains four fields: text content, corresponding feature vector, generation timestamp, and inherent importance.

[0053] Step 2: High-dimensional feature encoding of interactive observation states

[0054] In the During rounds of interaction, the agent receives the latest natural language commands from the user or observation logs of the external environment. .

[0055] Using a deep learning text embedding model for input text Perform high-dimensional space mapping to obtain the feature vector of the current state. :

[0056]

[0057] in, This represents the coordinates of the current task situation in the multidimensional semantic space, which will be used for subsequent intent matching.

[0058] Step 3: Working Memory Time Decay and Dynamic Archiving

[0059] This step is the core step in this invention to avoid the unlimited expansion of the large model context. Before the agent executes each round of actions, the system re-evaluates the weights of all historical records in the working memory.

[0060] This invention employs an exponential time decay function based on cognitive psychology, targeting the first... 1 record, current Retention weight at any time The calculation formula is as follows:

[0061]

[0062] in, Assign an initial base weight to this interaction record to the system. It is a natural constant (approximately 2.718). This indicates the time difference or round difference between the occurrence of the record and the current round. The preset decay rate hyperparameter for the system should be noted as follows: It's not a fixed value, but rather dynamically bound to the agent's current task type: when performing tasks with extremely high requirements for short-term context (such as frequent code modifications), the value is increased. The value is set to achieve rapid forgetting (short half-life); when performing long-term observation tasks (such as 24 / 7 log monitoring), it should be lowered. This is to achieve slow forgetting (long half-life).

[0063] When the retention weight of a certain record Falling below the system's preset archiving threshold When this happens, the system removes it from the working memory area and writes its text and corresponding feature vector into the context memory area for deep hibernation storage.

[0064] For example, in a coding task, variable definitions from 10 rounds ago are no longer used, their weight decays to below a threshold, and they are automatically archived into contextual memory. Only the function content being edited in the most recent rounds is retained in working memory.

[0065] Step 4: Non-linear gating evoked episodic memory

[0066] This step is the core mechanism of this invention to avoid catastrophic forgetting and resolve memory prioritization confusion. Before the agent generates an action, it checks whether the contextual memory contains dormant records valuable for the current decision.

[0067] Before the agent generates an action, the system uses a high-dimensional spatial indexing algorithm to retrieve contextual memories to recall historical experiences. To overcome the semantic omissions that are easily caused by traditional keyword-based matching (such as the BM25 algorithm), traditional retrieval relies solely on cosine similarity, which makes it difficult to distinguish between "relevant but outdated and useless" and "relevant and recently high-risk" memories. This embodiment adopts a multi-dimensional comprehensive recall scoring mechanism based on nonlinear semantic gating.

[0068] Let the current state feature vector be The first in the episodic memory bank Historical feature vectors for A 3D floating-point array, whose generation time is... The importance of static foundations is Define the overall wake-up score for this record. The calculation formula is as follows:

[0069]

[0070]

[0071] The parameter definitions and control logic are as follows:

[0072] and Let represent the magnitudes of the two vectors respectively, and let be the cosine similarity operator, representing the similarity between the current situation and historical memory in the core semantic direction.

[0073] The core semantic hard threshold is preset for the system.

[0074] For unit step gate function (when The output is 1 when the cosine similarity crosses a hard threshold, and 0 otherwise. This operator constructs a "non-linear semantic gate": it only outputs 1 when the cosine similarity crosses a hard threshold. At that time, the gating is activated, and the subsequent time decay operator... With inherent importance operator Only then is it activated to participate in the scoring; if the similarity does not meet the standard, the overall score of the entire memory is directly reduced to zero and it is blocked by the system.

[0075] This is a regulating factor used to balance the weighting ratio between relevance and inherent importance.

[0076] Comprehensive wake-up score of all records in the system's computing database The records with the highest scores and greater than 0 are extracted from the context memory area and added to the candidate set.

[0077] This nonlinear semantic gating mechanism, on the one hand, directly filters out all semantically irrelevant memories, greatly reducing the amount of computation; on the other hand, it prioritizes records that are "relevant, recent, and important" among relevant memories, avoiding the problem of outdated information interference caused by traditional methods that only look at similarity.

[0078] For example, when a user mentions "database connection" again, the system will recall the most recent database connection error and its solution, instead of using old configuration information that has been obsolete for 100 rounds.

[0079] Step 5: Multi-source context fusion and agent planning

[0080] The system uses a fixed template to concatenate structured strings from different levels of memory, constructing prompt words for the final input large language model:

[0081]

[0082] in, This indicates a structured approach. The system integrates a complete context that combines "absolute principles (core) + historical experience (scenario) + current situation (work)". The input is fed into a large language model. The large model performs logical reasoning based on this context and outputs the first... The specific action instructions to be executed for each step.

[0083] This three-layer context structure ensures that core constraints are not lost, incorporates relevant historical experience, and controls the overall context length. The large language model performs logical reasoning based on this complete context, outputting the first... The specific action instructions to be executed in a step, such as calling a tool, editing a piece of code, or returning a certain answer.

[0084] Step 6: Feedback on execution results and system status closure

[0085] The intelligent agent executes the actions generated in step 5 in a real software system or external environment and obtains the execution results, such as data returned by API calls, command line output, system error messages, file operation results, etc.

[0086] The system packages the execution instructions and corresponding results of this round into a new interaction record, assigns it an initial base weight, and pushes it to the tail of the working memory queue. The subsequent time rounds proceed as follows: Then return to step 2 to enter the next cycle, until the overall task objective is achieved or the user terminates the task.

[0087] Example Effect Verification

[0088] Comparative tests were conducted in long-cycle task scenarios involving multiple rounds of code debugging, with an average of 50 rounds per task.

[0089] Traditional sliding window method: task completion rate 58%, average token consumption 12k / round, early constraint forgetting rate 37%;

[0090] Pure vector retrieval method: task completion rate 72%, average token consumption 18k / round, outdated information interference rate 29%;

[0091] The method of this invention achieves a task completion rate of 91%, an average token consumption of 6k per round, a core constraint forgetting rate of 0, and an 83% reduction in invalid history recall rate.

[0092] Experimental results show that the method of the present invention can significantly improve planning accuracy, maintain a stable context length, avoid forgetting key information, and its overall performance is significantly better than that of existing methods in long-cycle tasks.

[0093] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for long-cycle task planning for intelligent agents based on nonlinear gating and time decay, characterized in that, Includes the following steps: Step 1: Construct a three-layer memory architecture, namely the core memory area that stores core constraints and does not participate in decay, the working memory area that uses a queue to store recent interactions, and the context memory area that uses vectors to store historical experience. Write the user's initial global constraints into the core memory area and keep them permanently in the large model context. Step 2: Receive new user instructions or environmental observations in each round, and use a text embedding model to perform high-dimensional space mapping on the input to obtain the current state feature vector; Step 3: Calculate the retention weight after time decay for each historical record in the working memory area, and remove records with retention weights lower than the archiving threshold from the working memory area and write them to the context memory area for storage. Step 4: Retrieve the context memory area based on the current state feature vector and adopt a non-linear semantic gating mechanism: only when the cosine similarity between a record and the current state exceeds a preset hard threshold, the comprehensive wake-up score is calculated by combining the time decay factor and inherent importance; records with similarity below the threshold are directly filtered out. Step 5: Select the episodic memory records with the highest comprehensive arousal scores, and combine the core memory, the selected episodic memory, and the current content of working memory according to a fixed template. Input the data into the large language model to deduce the current round of execution actions. Step 6: Execute the action in the environment, obtain the execution result, write the action and result as the latest record into the working memory area, increment the round by 1 and return to step 2 until the task is completed.

2. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 1, characterized in that, Step 1 specifically includes: Step 1.1: The core memory area stores the user's initial global constraints, safety criteria, and core objectives, assigns the highest scheduling weight, does not participate in any decay calculations, and permanently resides in the input context of the large model; Step 1.2: The working memory area adopts a first-in-first-out queue structure to store the interaction logs of the most recent rounds, the current subtask status, and short-cycle environmental feedback; Step 1.3: The context memory area is implemented using a vector database built on the approximate nearest neighbor algorithm. It persistently stores historical interaction records and trial-and-error experiences archived from the working memory area. Each record contains four fields: text content, feature vector, generation timestamp, and inherent importance.

3. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 1, characterized in that, In step 2, a deep learning text embedding model is used to embed the input text. Perform high-dimensional space mapping to obtain the feature vector of the current state. : ； Among them, feature vector This represents the coordinates of the current task situation in the multidimensional semantic space, which will be used for subsequent intent matching.

4. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 1, characterized in that, In step 3, the first [item] in working memory 1 record, which is in the current Retention weight at any time The calculation formula is: ； in, Assign an initial base weight to this interaction record to the system. It is a natural constant. This indicates the time difference or round difference between the occurrence of the record and the current round. The preset decay rate hyperparameter for the system, It's not a fixed value; it's dynamically bound to the agent's current task type: when performing tasks with extremely high short-term context requirements, the value is increased. Values ​​that promote rapid forgetting, i.e., short half-life; when performing long-term observation tasks, adjust the value to achieve this. This allows for slow forgetting, i.e., a long half-life; When the retention weight of a certain record Falling below the system's preset archiving threshold When this happens, the system removes it from the working memory area and writes its text and corresponding feature vector into the context memory area for deep hibernation storage.

5. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 1, characterized in that, Step 4 specifically includes: Let the current state feature vector be The first in the episodic memory bank Historical feature vectors for A 3D floating-point array, whose generation time is... The importance of static foundations is Define the overall wake-up score for this record. The calculation formula is as follows: ； ； The parameter definitions and control logic are as follows: and Let represent the magnitudes of the two vectors respectively, and let be the cosine similarity operator, representing the similarity between the current situation and historical memory in the core semantic direction; The system has a preset core semantic hard threshold; For unit step gate function, when The output is 1 when the condition is met, and 0 otherwise. This is a regulating factor used to balance the weighting ratio between relevance and inherent importance; Comprehensive wake-up score of all records in the system's computing database The records with the highest scores and greater than 0 are extracted from the context memory area and added to the candidate set.

6. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 5, characterized in that: The cosine similarity operator constructs a nonlinear semantic gate: Only when the cosine similarity crosses the hard threshold At that time, the gating is activated, and the subsequent time decay operator... With inherent importance operator Only then is it activated to participate in the scoring; if the similarity does not meet the standard, the overall score of the entire memory is directly reduced to zero and it is blocked by the system.

7. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 1, characterized in that, In step 5, the final prompt words for the input large language model are constructed as follows: ； Here, ⨁ represents structured assembly, where the system combines absolute principles, historical experience, and the complete context of the current situation. The input is fed into a large language model, which performs logical reasoning based on the context and outputs the first... The specific action instructions to be executed for each step.

8. The agent-based long-cycle task planning method based on nonlinear gating and time decay according to claim 1, characterized in that, In step 6, the agent executes the generated action in the external environment or software system and obtains execution feedback, including API return results, system errors, and state changes. Package the action instructions and execution results of this round into a new interaction record, assign it an initial base weight, and push it to the end of the working memory queue; Round Promote as Then return to step 2 to enter the next cycle until the overall task objective is achieved or the termination condition is triggered.

Images