A processing method and electronic device

Abstract

The application discloses a processing method and an electronic device. The method comprises the following steps: acquiring a state parameter corresponding to execution of a first task, and attribute values of each first candidate memory in a memory pool on a plurality of attribute parameters; acquiring execution feedback information of the first task; adjusting a first weight coefficient corresponding to each of the plurality of attribute parameters according to the state parameter, the execution feedback information, and the attribute values of each first candidate memory on the plurality of attribute parameters, to obtain a second weight coefficient; and processing the attribute values of each second candidate memory in the memory pool on the plurality of attribute parameters based on the second weight coefficient, to determine an attribute factor of each second candidate memory.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more particularly to a processing method and an electronic device. Background Technology

[0002] Existing memory management schemes for intelligent agents are typically based on fixed time windows, semantic similarity retrieval, or static weighted scoring mechanisms. The core flaw of these schemes lies in the lack of adaptive optimization capabilities in memory retrieval and eviction strategies. They cannot dynamically adjust the evaluation criteria of memories based on the actual performance feedback of the task, making it difficult to stably retain the optimal memory set under different task scenarios. Summary of the Invention

[0003] The technical solution provided in this application is as follows: The first aspect of this application provides a processing method, including: Obtain the status parameters corresponding to the execution of the first task, as well as the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters; Obtain the execution feedback information of the first task; At least based on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients; Based on the second weighting coefficient, the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters are processed to determine the attribute factors of each second candidate memory.

[0004] In one possible implementation, adjusting the first weight coefficients corresponding to each of the plurality of attribute parameters based at least on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the plurality of attribute parameters to obtain the second weight coefficients includes: Obtain the attribute factors of each of the first candidate memories; the attribute factors of the first candidate memories are obtained by processing the attribute values ​​of the first candidate memories on multiple attribute parameters based on the first weight coefficient. Based on the state parameters, the execution feedback information, the attribute values ​​of each first candidate memory on multiple attribute parameters, and the attribute factors of each first candidate memory, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients.

[0005] In one possible implementation, obtaining the execution feedback information of the first task includes: Obtain at least one of the following for the first task: task success rate, memory pool storage overhead, retrieval time, and tool call success rate; A reward value is generated based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate.

[0006] In one possible implementation, generating a reward value based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool invocation success rate includes: Obtain the first reward weight corresponding to the task success rate, the second reward weight corresponding to the memory pool storage overhead, the third reward weight corresponding to the retrieval time, and the fourth reward weight corresponding to the tool call success rate; The reward value is obtained by processing the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate according to the first reward weight, the second reward weight, the third reward weight, and the fourth reward weight respectively. The first reward weight and the fourth reward weight are used to generate positive rewards for the task success rate and the tool call success rate, while the second reward weight and the third reward weight are used to generate negative rewards for the memory pool storage overhead and the retrieval time.

[0007] In one possible implementation, obtaining the execution feedback information of the first task includes: Obtain the immediate reward, process reward, and final state reward corresponding to the first task; The immediate reward, the process reward, and the final state reward are weighted to obtain the total reward value. The instant reward is determined based on at least one of the following: the adoption rate of the memory retrieval results corresponding to the first task, the redundancy of the retrieval results, and the relevance of the retrieval results. The process reward is determined based on at least one of the tool call success rate corresponding to the first task, repeated call penalty, and failure experience avoidance reward. The final state reward is determined based on the task success rate and memory efficiency reward corresponding to the first task.

[0008] In one possible implementation, obtaining the state parameters corresponding to the execution of the first task includes at least one of the following: Obtain the type of the first task, which will be used as the status parameter for executing the first task; The occupancy rate of the memory pool is determined and used as a status parameter for executing the first task.

[0009] In one possible implementation, determining the occupancy rate of the memory pool includes any of the following: The ratio of the used storage space to the total storage space of the memory pool is determined as the occupancy rate of the memory pool; The ratio of the number of memories already stored in the memory pool to the maximum number that can be stored is determined as the occupancy rate of the memory pool.

[0010] In one possible implementation, the processing method further includes: At least based on the state parameters, the execution feedback information, and the attribute values ​​of each first candidate memory on multiple attribute parameters, an update operation is performed on a portion of the first candidate memories in the memory pool to obtain second candidate memories.

[0011] In one possible implementation, the plurality of attribute parameters includes any of the following: The first attribute parameter is used to reflect the storage duration of the corresponding candidate memory in the memory pool; The second attribute parameter is used to reflect the semantic similarity between the corresponding candidate memory and the corresponding task in the memory pool; The third attribute parameter is used to reflect the reliability of the data source of the corresponding candidate memory in the memory pool; The fourth attribute parameter is used to reflect the frequency with which the corresponding candidate memory in the memory pool is called in historical tasks; The fifth attribute parameter is used to reflect the risk level of the corresponding task failing to execute after the corresponding candidate memory in the memory pool is lost.

[0012] Another aspect of this application provides an electronic device, comprising: processor; Memory; The memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the following method steps: Obtain the status parameters corresponding to the execution of the first task, as well as the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters; Obtain the execution feedback information of the first task; At least based on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients; Based on the second weighting coefficient, the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters are processed to determine the attribute factors of each second candidate memory. Attached Figure Description

[0013] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale.

[0014] Figure 1 A flowchart illustrating a processing method provided in Embodiment 1 of this application; Figure 2 This is a flowchart illustrating a processing method provided in Embodiment 2 of this application; Figure 3 This is a flowchart illustrating a processing method provided in Embodiment 3 of this application; Figure 4 This is a flowchart illustrating a processing method provided in Embodiment 4 of this application; Figure 5 This is a flowchart illustrating a processing method provided in Embodiment 6 of this application; Figure 6 This is a schematic diagram of a system architecture provided for this application. Detailed Implementation

[0015] The embodiments of this application are described below with reference to the accompanying drawings. The terminology used in the implementation section of this application is for explaining specific embodiments only and is not intended to limit the scope of this application.

[0016] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.

[0017] The terms "first," "second," etc., used in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms can be used interchangeably where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of units is not necessarily limited to those units, but may include other units not explicitly listed or inherent to those processes, methods, products, or apparatuses.

[0018] In the embodiments of this application, reference is made to Figure 1 This is a flowchart illustrating a processing method provided in Embodiment 1 of this application, as shown below. Figure 1 As shown, the method may include, but is not limited to, the following steps: Step S101: Obtain the status parameters corresponding to the execution of the first task, and the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters.

[0019] Executing the first task requires retrieving relevant memories from the memory pool as context for execution. Memory retrieval is achieved through a comprehensive evaluation of multiple attribute parameters. The actual reference value of a single memory for the first task cannot usually be determined by a single dimension alone; rather, it requires a comprehensive assessment of multiple factors. For example, a semantically highly relevant memory may be rejected due to an unreliable source, while a memory stored for a long time but containing crucial lessons learned from failure should not be easily ignored. Therefore, each memory is evaluated based on multiple attribute parameters. Each attribute parameter characterizes a specific characteristic of the memory from a different perspective, and each attribute parameter corresponds to a weighted coefficient, representing the relative importance of that dimension in the comprehensive evaluation.

[0020] During retrieval, for each first candidate memory in the memory pool, the attribute value of the first candidate memory on the corresponding attribute parameter is obtained. Then, each attribute parameter is weighted and summed with its corresponding weight coefficient to obtain the attribute factor of the first candidate memory. The higher the attribute factor, the more valuable the first candidate memory is considered after multi-dimensional comprehensive evaluation under the current weight coefficient configuration. Based on this, several memories with higher attribute factors are selected from the first candidate memories as reference contexts for performing the first task.

[0021] After the first task is completed, its result, whether successful or not, objectively constitutes a practical test of whether the current weight coefficient configuration is reasonable. Task success indicates, to some extent, that the current weight coefficient configuration is well-suited to the current task requirements; task failure may mean that the current weight coefficient configuration is flawed, causing some key memories that should have been prioritized for retrieval to be ignored or excluded from the search scope.

[0022] However, simply concluding that a task succeeded or failed is insufficient to pinpoint which dimension's weight configuration was flawed, and therefore cannot guide the direction in which the weight coefficients should be adjusted. To make the adjustments more targeted, additional information is needed to determine how the relative importance of each memory dimension should be reassessed in the specific scenario following the execution of this task. Therefore, this step obtains the state parameters corresponding to the first task.

[0023] The state parameters describe the objective scenario formed after the first task is completed, influencing the relative importance of each attribute parameter in the next retrieval. After the first task is completed, conclusive feedback on the effect of the current weight coefficient configuration is obtained, but it cannot directly indicate which attribute parameter's weight coefficient needs adjustment, or whether it should be increased or decreased. Different task execution results, under different task scenarios, may lead to drastically different directions for correcting the weight coefficients corresponding to each attribute parameter. The state parameters provide a basis for distinguishing different task scenarios, ensuring that weight coefficient adjustments are based on the actual scenario after the first task is completed, rather than blind attempts detached from the scenario.

[0024] The first candidate memory can be understood as each memory that actually exists in the memory pool after the first task is completed. For each first candidate memory, this step obtains the attribute values ​​on multiple attribute parameters.

[0025] When a memory is stored in the memory pool, its various attribute parameters are assigned initial values ​​based on its source, content, and the context of the first task. For example, the initial value of its time base parameter is determined by the memory's creation time, the initial value of its association parameter is determined by the degree of correlation between the memory content and the first task, and the initial value of its credibility parameter is determined by the reliability of the memory's source. During the execution of subsequent tasks, the attribute parameters of each memory are continuously updated over time and as tasks are executed. After the first task is completed, the attribute values ​​of each first candidate memory are read from the memory pool to accurately reflect the current state of the memories in the pool.

[0026] Step S102: Obtain the execution feedback information of the first task.

[0027] In the process of determining attribute factors through attribute parameters and weighting coefficients, the quality of memory filtering is directly affected by the configuration of the weighting coefficients. If the weighting coefficients are configured properly, truly valuable memories can be filtered out to assist task execution; conversely, key memories may be missed or irrelevant memories may be introduced, leading to a decline in task execution performance.

[0028] However, state parameters and attribute parameters alone are insufficient to determine the actual effect of the current weighting coefficient configuration. State parameters reflect the objective scenario after task execution, and attribute parameters reflect the individual performance of each memory segment across different dimensions; however, neither includes information about how the current weighting coefficient configuration actually performed in this task. Therefore, execution feedback information is also needed.

[0029] Execution feedback information, based on the actual results of task execution, provides objective feedback signals on the effectiveness of the current weight coefficient configuration. Execution feedback information can reflect multiple aspects, such as whether the task was successfully completed, the efficiency of the memory retrieval process, and the success rate of external tool calls.

[0030] For example, if the feedback indicates that the task is performed well, it means that the current weighting coefficient configuration is reasonable in the current context and can be maintained or only slightly adjusted. If the feedback indicates that the task is not performed well, it means that the current weighting coefficient configuration may need to be significantly revised in order to obtain better memory filtering results in the next task.

[0031] Step S103: Based at least on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, adjust the first weight coefficients corresponding to each of the multiple attribute parameters to obtain the second weight coefficients.

[0032] The first weighting coefficient can be an initial weighting coefficient that has not yet been adjusted for a specific task, or it can be a weighting coefficient obtained by gradually adjusting the weighting coefficients based on the state parameters, execution feedback information, and attribute values ​​of each candidate memory in the memory pool of each task after the execution of each previous task. The initial weighting coefficient is the weighting coefficient used by the system before accumulating any task execution experience. It is used for memory retrieval when first encountering a task, and it can be pre-set based on historical data or prior knowledge.

[0033] The reason for continuously updating the weighting coefficients after each task execution is that the relative importance of each attribute parameter varies across different task scenarios. For example, in some tasks, the freshness of a memory may be more important than the risk of forgetting. In other tasks, the risk of forgetting is far more important than freshness. If the weighting coefficients remain fixed, the ranking criteria for memories will be exactly the same regardless of the task scenario, inevitably leading to truly critical memories in some scenarios being unable to be retrieved first due to their low weight.

[0034] Before the first task is executed, the actual effect of the first weighting coefficient in that task scenario is unknown. After the first task is completed, new information that was not available before its execution is obtained: the execution feedback information reflects the actual execution effect of the first task under the first weighting coefficient; the state parameters reflect the objective scenario after the first task is completed; and the attribute values ​​of the first candidate memory on multiple attribute parameters reflect the actual attributes of the first candidate memory. Based on this new information, the first weighting coefficient is adjusted so that the adjusted second weighting coefficient more accurately reflects the relative importance of each attribute parameter in the current scenario than the unadjusted first weighting coefficient.

[0035] During the adjustment process, the state parameters, execution feedback information, and attribute values ​​of the first candidate memory can be used as inputs to adjust the first weight coefficients through a reinforcement learning strategy, resulting in the adjusted weight coefficients corresponding to each attribute parameter, i.e., the second weight coefficients. Other optimization algorithms can also be used to calculate the corrected weight coefficients based on the above three types of information; this embodiment does not impose any restrictions on this approach.

[0036] Through continuous iteration, which adjusts the weighting based on new information after each task is completed, the weighting coefficients are continuously optimized as task experience accumulates, gradually approaching the optimal configuration for various task scenarios.

[0037] Step S104: Based on the second weighting coefficient, process the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters, and determine the attribute factors of each second candidate memory.

[0038] Once the first weighting coefficient is adjusted, the attribute factors of all existing memories will be automatically recalculated according to the new weighting coefficient, i.e., the second weighting coefficient. Newly stored memories will also be evaluated according to the new weighting coefficient. This allows the experience accumulated from previous tasks, including the first task, to be applied to the memory retrieval process in subsequent tasks, thus optimizing memory pool management.

[0039] When the second task is received after the first task, the memories in the memory pool may have changed. This is because new memories may have been added to the memory pool during the execution of the first task, or some attribute parameters of existing memories may have been updated over time or due to task execution. These memories that exist in the memory pool when the second task arrives after the first task has been completed are called second candidate memories.

[0040] For the second task, the latest weighting coefficients obtained after this adjustment, i.e., the second weighting coefficients, will be used to weight the attribute values ​​of each second candidate memory across multiple attribute parameters, calculating the attribute factor of each second candidate memory. The attribute factor of the second candidate memory represents the comprehensive value of each memory relative to the new task request under the new weighting coefficient configuration.

[0041] Then, based on the second attribute factors of each second candidate memory, the target memory for performing the second task can be determined from among the multiple second candidate memories. In one optional implementation, the second candidate memories can be sorted from highest to lowest according to their attribute factors, and a predetermined number of the top-ranked second candidate memories can be determined as the target memories. In another optional implementation, an attribute factor threshold can be preset, and second candidate memories with attribute factors greater than the threshold can be determined as the target memories. The target memory is the memory content referenced when performing the second task.

[0042] It should be noted that the attribute values ​​of the second candidate memory on multiple attribute parameters follow the same update mechanism as those of the first candidate memory. Both are assigned initial values ​​when stored in the memory pool and are continuously updated over time and with task execution. Therefore, although the second candidate memory and the first candidate memory physically belong to the same memory pool, their attribute values ​​may differ.

[0043] In this embodiment, the introduction of state parameters and execution feedback information allows the adjustment of weight coefficients to be based on the objective scenario and actual execution effect after the first task is completed, rather than blindly trying things out of context. The attribute values ​​of the first candidate memory on multiple attribute parameters provide the actual attributes of each memory in the current memory pool for adjustment, further improving the targeting and accuracy of the adjustment. Through continuous iteration based on state parameters, execution feedback information, and attribute values ​​on multiple attribute parameters after each task execution, the weight coefficients are continuously optimized with the accumulation of task experience, gradually adapting to the changes in the relative importance of each attribute parameter under different task scenarios, thereby improving the accuracy of memory retrieval and the task execution effect in subsequent tasks.

[0044] In one optional implementation, the plurality of attribute parameters may include any of the following: The first attribute parameter is used to reflect the storage duration of the corresponding candidate memory in the memory pool.

[0045] Generally, the shorter the storage time of a memory, the closer the information it reflects is to the current state, and the higher its reference value is for tasks that require real-time or time-sensitive information; while the longer the storage time of a memory, the more outdated the information it reflects may still contain some lessons that have long-term value.

[0046] The specific value of the first attribute parameter can be determined by the time decay function. The input to the time decay function is the storage duration, and the output is a value between 0 and 1. For example, the time decay function can be set to a monotonically decreasing function, so that memories with shorter storage times receive higher values ​​for the first attribute parameter, and memories with longer storage times receive lower values ​​for the first attribute parameter. The specific decay rate can be set according to the actual application scenario; for example, linear decay, exponential decay, or step decay can be used.

[0047] The second attribute parameter is used to reflect the semantic similarity between the corresponding candidate memory and the corresponding task in the memory pool.

[0048] The higher the semantic similarity, the closer the content of the memory is to the needs of the current task, and the higher the reference value of the memory for task execution.

[0049] The value of the second attribute parameter can be determined by calculating the similarity between the semantic vector of the memory content and the semantic vector of the task description. For example, the memory content and the task description can be input into a semantic encoding model to obtain their respective semantic vector representations. Then, the cosine similarity or Euclidean distance between the two semantic vectors can be calculated, and the result can be used as the value of the second attribute parameter. The semantic similarity value usually ranges from 0 to 1, with a value closer to 1 indicating greater semantic similarity and a value closer to 0 indicating less semantic relevance.

[0050] The third attribute parameter is used to reflect the reliability of the data source for the corresponding candidate memory in the memory pool.

[0051] Different sources of information have different levels of reliability. For example, memories from authoritative databases or verified tools are generally more reliable, while memories from unverified user input or low-quality online sources are less reliable.

[0052] The attribute value of the third attribute parameter can be preset according to the type of data source. For example, different reliability scores can be assigned to different types of data sources: memories from official documents or authoritative knowledge bases can be assigned a higher reliability score (e.g., 0.9), memories from verified tool call results can be assigned a medium-to-high reliability score (e.g., 0.7), and memories from unverified user input can be assigned a lower reliability score (e.g., 0.3).

[0053] Furthermore, the attribute value of the third attribute parameter can be dynamically adjusted based on the actual usage effect during subsequent task execution. For example, if a memory from a low-reliability source proves to be accurate in multiple tasks, the attribute value of its third attribute parameter can be appropriately increased.

[0054] The fourth attribute parameter is used to reflect the frequency with which the corresponding candidate memory in the memory pool is called in historical tasks.

[0055] Frequently accessed memories usually indicate that they have reference value in multiple task scenarios and are useful information that has been repeatedly verified; while rarely accessed memories may be peripheral information, and their actual value needs to be confirmed.

[0056] The value of the fourth attribute parameter can be determined by statistically analyzing the number of times a memory is retrieved in historical tasks. For example, a counter can be maintained for each memory, incrementing by 1 each time the memory is retrieved and used for task execution. The value of the fourth attribute parameter is the current value of that counter or its normalized result. It is important to note that the frequency of access should consider the time factor; earlier access records may contribute differently to the current reference value than more recent ones. Therefore, a weighted statistical method with time decay can also be used.

[0057] The fifth attribute parameter is used to reflect the risk level of the corresponding task failing to execute after the corresponding candidate memory in the memory pool is lost.

[0058] A memory may not have high semantic similarity or high access frequency, but if it contains key security rules, important lessons learned from failures, or unique user preference information, then its risk of being forgotten is very high, because once lost, subsequent tasks may suffer serious losses as a result.

[0059] The value of the fifth attribute parameter can be estimated based on the characteristics of the content being remembered and its impact on the task. For example, memories containing high-risk event records (such as payment failures, security vulnerabilities, and compliance violations) can be assigned a higher risk of forgetting; memories containing unique information (such as user personalization settings and specific business processes) can also be assigned a higher risk of forgetting; while general memories that can be retrieved from other sources can be assigned a lower risk of forgetting.

[0060] The value of the fifth attribute parameter can be initially assigned when it is stored in memory, and can be dynamically updated according to the actual usage during subsequent task execution.

[0061] After obtaining the attribute values ​​of each candidate memory on the above-mentioned multiple attribute parameters, the attribute factors of each candidate memory can be calculated using the following formula:

[0062] in, This represents the attribute factor of the i-th candidate memory. The higher the attribute factor, the more valuable the memory is considered to be under the current weighting coefficient configuration.

[0063] This represents the storage duration of the i-th candidate memory. This represents the attribute value of the first attribute parameter obtained after normalizing the storage duration, which is the output value of the time decay function. This represents the weighting coefficient corresponding to the first attribute parameter, used to control the relative importance of storage duration in the overall evaluation.

[0064] Let represent the semantic similarity between the i-th candidate memory and the current task. This represents the attribute value of the second attribute parameter obtained after normalizing the semantic similarity. This represents the weight coefficient corresponding to the second attribute parameter, which is used to control the relative importance of semantic similarity in the comprehensive evaluation.

[0065] This indicates the reliability of the data source for the i-th candidate memory. This represents the attribute value of the third attribute parameter obtained after normalizing the reliability level. This represents the weighting coefficient corresponding to the third attribute parameter, used to control the relative importance of the reliability of the data source in the comprehensive evaluation.

[0066] This indicates the frequency with which the i-th candidate memory is invoked in the historical task. This represents the attribute value of the fourth attribute parameter obtained after normalizing the frequency. This represents the weighting coefficient corresponding to the fourth attribute parameter, used to control the relative importance of access frequency in the overall evaluation.

[0067] This indicates the risk level of task failure caused by the loss of the i-th candidate memory. This represents the value of the fifth attribute parameter obtained after normalizing the risk level. This represents the weighting coefficient corresponding to the fifth attribute parameter, used to control the relative importance of the risk of forgetting in the comprehensive assessment.

[0068] In the above formula, five attribute parameters characterize the value of a memory from different dimensions: storage duration reflects the freshness of the memory, semantic similarity reflects the degree of matching between the memory and the current task, data source reliability reflects the credibility of the memory, access frequency reflects the historical usage of the memory, and forgetting risk reflects the potential cost of memory loss. By weighting and summing the attribute values ​​of these five dimensions with their corresponding weight coefficients, a comprehensive evaluation score for a memory can be obtained, namely the attribute factor.

[0069] Weighting coefficient to These parameters are dynamically adjustable, determining the relative importance of each attribute parameter in the overall evaluation. For example, if semantic similarity is more important than storage duration in the current task scenario, then... Set to be more Larger; if the risk of forgetting is the most critical factor in the current task scenario, then it can be... Set them to the maximum. By adjusting these weighting coefficients, the evaluation criteria for memory can be adapted to the actual needs of different task scenarios.

[0070] It should be noted that the above formula applies to the calculation of attribute factors for both the first and second candidate memories. The difference between the two lies in the weighting coefficients and attribute values ​​used for the attribute parameters.

[0071] For the calculation of attribute factors of the first candidate memory, to The first weight coefficient corresponds to each attribute parameter; the attribute factor calculation for the second candidate memory is as follows: to The second weighting coefficient corresponding to each attribute parameter.

[0072] In one optional implementation, the initial weighting coefficients can be determined using a cold start initialization mechanism. This mechanism addresses the issue that when encountering a new task type or user scenario for the first time, the weighting coefficients cannot be adjusted based on historical task feedback due to a lack of prior task execution experience. The cold start initialization mechanism may include the following three stages.

[0073] Phase 1: Meta-learning offline pre-training.

[0074] Before deployment, offline pre-training was performed using a large amount of historical multi-task data. A cross-task general baseline weight was obtained through a meta-learning algorithm (such as the Model Independent Meta-Learning Algorithm, MAML). This baseline weight serves as the initialization starting point for the weight coefficients in all new task scenarios, denoted as: α_default = [α1°, α2°, α3°, α4°, α5°]

[0075] Here, α1° to α5° represent the baseline weight coefficients corresponding to the five attribute parameters, respectively.

[0076] The training data used in Phase 1 consists of task trajectory data. Each task trajectory sample includes: a task type label (task_type), step-by-step execution records, the final result of the task (success / fail), and the optimal weight label (optimal_alpha) corresponding to that trajectory. The step-by-step execution records include which memories were retrieved, what actions were performed, and the tool call results for each step. Task trajectory data is primarily sourced from the historical operation logs of existing intelligent agent systems; when historical data is unavailable, a simulator can be used to generate data covering multiple task types, typically a few dozen trajectories per task type are sufficient to meet the meta-learning requirements.

[0077] The optimal weight labeling can be obtained through a reverse inference method: for a successful trajectory, the implicit preferences of each attribute dimension are inferred from the actual memory retrieval sequence used to obtain the optimal weight labeling for that trajectory. When engineering conditions permit, grid search exhaustive method can be used to further refine the labeling of some trajectories to improve the labeling quality.

[0078] Phase Two: Task Type Similarity Transfer.

[0079] During the online phase, when a new task type T_new is encountered for the first time, the similarity sim(T_new, T_k) between the feature vector of the new task type and the feature vectors of each historical type T_k in the task type library is calculated. The task type library is automatically generated after the training in Phase 1, where each task type corresponds to a record containing task feature vectors (such as average number of steps, tool call frequency, task cycle, historical failure rate, etc.) and the weights of that type after the reinforcement learning converges.

[0080] Based on the similarity level, the initial weight coefficient α_init is determined using a hierarchical judgment method: If sim(T_new, T_k)≥ θ_high (e.g., 0.8), then it is determined to be a task of the same type, and the convergence weight of the nearest neighbor history type is directly used as the initial weight, that is: α_init = α_k; If θ_low ≤ sim(T_new, T_k) < θ_high (e.g., 0.4~0.8), then the initial weights are calculated according to the weighted migration formula: α_init(T_new) = (1-η)·α_default + η·α_k; Where η is the normalized similarity coefficient.

[0081] If sim(T_new, T_k) < θ_low, it is determined to be an out-of-distribution (OOD) new task type. No weight transfer is performed, and the baseline weights are directly used as the initial weights: α_init = α_default, and it is switched to the high exploration starting point and slow decay mode of Phase 3.

[0082] Among them, the high threshold θ_high and the low threshold θ_low can be determined by using the reserved validation task set in the first stage of pre-training, statistically analyzing the relationship curve between task feature similarity and convergence speed or final reward after policy transfer, and taking the inflection point of the curve as the threshold, rather than relying solely on human experience.

[0083] The task type library is not built once and remains unchanged, but is continuously updated as the system runs online. This includes both the inclusion of new task types and the continuous updating of known types.

[0084] For the initial cold start convergence of a new task type: After the new task type T_new converges after Phase 3 training (e.g., the task reward variance is lower than the set threshold for M consecutive times), the task type library is updated according to the following rules, so that the coverage of the library continues to expand as the system runs online, rather than remaining at the result of a one-time build before deployment: If the similarity between its feature vector and an existing type in the library is ≥θ_high, then the convergence weight of the type is updated by moving average: α_k ← (1-μ)·α_k + μ·α_converged(T_new); Where μ is the update rate, used to prevent library fragmentation.

[0085] If the similarity with all types in the library is less than θ_high, then it is written as a new type entry into the task type library.

[0086] If a known type is no longer updated, its α_k will gradually become outdated as the task distribution of that type shifts. Therefore, this invention adds a second triggering condition for the continuous updating of known types: For known task types that are continuously refreshed: If the average reward of the most recent k tasks of the same type deviates continuously from the historical average of the type (exceeding the set threshold, indicating that the task distribution may drift), the weight of the type in the library will be refreshed periodically according to the same moving average formula α_k ← (1-μ)·α_k + μ·α_converged to avoid the weight configuration of known types becoming fixed.

[0087] Phase 3: Adaptive exploration rate decay.

[0088] During the online cold start period (e.g., the first N tasks, where N typically ranges from 10 to 50), the exploration rate ε linearly decreases from 0.8 to 0.1, and the learning rate is set to 2-3 times that of the stable period to accelerate the weight fitting of the new task distribution; after N tasks, it automatically switches to the standard RL update rhythm. For new task types that are determined to be OOD (out of distribution) in Phase 2, the starting point of the exploration rate is appropriately increased and the decay is appropriately slowed down to obtain more thorough exploration.

[0089] In Phase 3, the agent automatically generates a reinforcement learning standard quadruple (current state) after each action. Execution of actions Receive rewards Next state The high exploration rate during the cold start phase ensures that the first N tasks will cover a variety of weight coefficient combinations, providing sufficient sample diversity for fast convergence.

[0090] To facilitate understanding of the above mechanism, three examples of new task processing are given below. It should be noted that the example parameters are for illustrative purposes only, and the actual values ​​should be based on engineering calibration.

[0091] Suppose there is already one record in the task type library: Task type T1 = "Document Summary", convergence weight α_k = [0.3, 0.4, 0.1, 0.1, 0.1].

[0092] Example 1: A completely new task type, such as "multi-agent collaborative scheduling": The calculated highest similarity with all types in the library is sim=0.25, which is lower than θ_low, so it is judged as OOD. Weights are not transferred; instead, they are directly initialized with α_default, and the process transitions to the high exploration starting point and slow decay mode of Phase 3. After training convergence, because the similarity with all types in the library is still lower than θ_high, it is added as a new entry to the task type library.

[0093] Example 2: Similar to known types, such as "Contract Summary," the structure is similar to "Document Summary," but with higher risk sensitivity. The calculated sim(T_new, T1) = 0.65, falling within the interval [θ_low, θ_high). It is initialized using the weighted migration formula α_init = (1-η)·α_default + η·α_k(T1), and then enters stage three for online fine-tuning. After convergence, if the similarity to all types in the library is still lower than θ_high, it is written as a new, independent entry and not merged with T1.

[0094] Example 3: Given the recurrence of a known task type, the "Document Summary" task arrives again. sim(T_new, T1) is calculated to be 0.9, reaching θ_high, and α_init=α_k(T1) is used for initialization. Subsequently, the average reward of recent tasks of this type is continuously monitored for any persistent deviation from the historical average: if there is no significant deviation, α_k remains unchanged, and subsequent tasks of the same type are directly reused; if a persistent deviation occurs (e.g., changes in business scenarios leading to increased task difficulty), the sliding average refresh rule in the aforementioned "online update of the task type library" is triggered, and α_k is gradually updated towards the new convergence weight direction.

[0095] The training data formats and sources required for the three stages are as follows: The data in Phase 1 is the task execution trajectory, which has already been explained in Phase 1 and will not be repeated here.

[0096] Data sources: Prioritize using historical operation logs from existing agent systems; when no historical data is available, use a simulator to generate data covering multiple task types. A few dozen trajectories for each task type are sufficient to meet the meta-learning requirements.

[0097] Phase 2 data is a task type library, automatically generated after Phase 1 training. Each task type corresponds to one record, containing a task feature vector (average number of steps, tool call frequency, task cycle, historical failure rate, etc.) and the weight α after RL convergence for that type. When a new task arrives, the cosine similarity between the feature vector and each type in the library is calculated, and the task is migrated or added as a new entry according to the aforementioned similarity grading threshold. This library is continuously and incrementally updated as the online system runs.

[0098] Phase 3 data consists of online quadruples, which are generated in real time after each action performed by the agent, including the current state. Execution of actions Receive rewards Next state ′, write to the experience replay pool. Phase 3 does not require prior data preparation.

[0099] It should be noted that the cold start initialization mechanism described above is only one optional method for determining the initial weight coefficients. In actual implementation, θ_high, θ_low, and the sliding window size... The deviation judgment threshold, exploration rate, and other parameters can all be configured according to actual engineering needs; this embodiment does not impose specific restrictions on them. The similarity, reward, and other values ​​in the above examples are merely illustrative, used to illustrate how the three-tier judgment rules and the two types of library update triggering conditions work together. It is recommended that specific values ​​be determined after calibration based on actual historical data.

[0100] As another optional embodiment of this application, refer to Figure 2This is a flowchart illustrating a processing method provided in Embodiment 2 of this application. This embodiment is mainly an implementation of step S103 in Embodiment 1, such as... Figure 2 As shown, the specific steps may include, but are not limited to, the following: Step S1031: Obtain the attribute factors of each of the first candidate memories; the attribute factors of the first candidate memories are obtained by processing the attribute values ​​of the first candidate memories on multiple attribute parameters based on the first weight coefficient.

[0101] While the status parameters, execution feedback information, and the attribute values ​​of the first candidate memory on multiple attribute parameters can reflect the objective scene after the execution of the first task, the execution effect of the first task, and the actual values ​​of each memory in the memory pool on each attribute parameter, they do not directly reflect the ranking results of each first candidate memory under the configuration of the first weight coefficient.

[0102] Without the ranking results of each first candidate memory, when the execution feedback information shows that the first task performed poorly, we can only know that the execution was poor, but we cannot determine whether the poor performance was caused by a ranking error. That is, we cannot determine whether a key memory that should have been ranked higher was ranked lower, or an irrelevant memory that should have been ranked lower was ranked higher. As a result, we cannot determine whether the current first weight coefficient configuration is reasonable, let alone which weight coefficient needs to be adjusted, or whether it should be adjusted by increasing or decreasing it. The adjustment of the weight coefficient will lack specific direction and can only be done blindly.

[0103] Based on this, this embodiment further introduces the attribute factor of the first candidate memory as the basis for adjusting the weight coefficient, based on embodiment 1.

[0104] Step S1032: Based on the state parameters, the execution feedback information, the attribute values ​​of each first candidate memory on multiple attribute parameters, and the attribute factors of each first candidate memory, adjust the first weight coefficients corresponding to each of the multiple attribute parameters to obtain the second weight coefficients.

[0105] In the specific adjustment process, the state parameters, execution feedback information, attribute values ​​of the first candidate memory, and attribute factors of the first candidate memory can be used as inputs. The first weight coefficients can be adjusted through reinforcement learning strategies or other optimization algorithms to obtain the weight coefficients corresponding to each attribute parameter after adjustment, i.e., the second weight coefficients.

[0106] In this embodiment, the attribute factor reflects the ranking result of each first candidate memory under the first weight coefficient configuration. By introducing the attribute factor, the adjustment process can know the actual ranking of each first candidate memory under the current first weight coefficient configuration, making the adjustment of the weight coefficient more specific, thereby improving the targeting and accuracy of the weight coefficient adjustment. This makes the adjusted second weight coefficient more in line with the actual memory needs of the current task scenario, and thus improves the accuracy of memory retrieval and the task execution effect in subsequent tasks.

[0107] As another optional embodiment of this application, refer to Figure 3 This is a flowchart illustrating a processing method provided in Embodiment 3 of this application. This embodiment is mainly an implementation of step S102 in Embodiment 1, such as... Figure 3 As shown, the specific steps may include, but are not limited to, the following: Step S1021: Obtain at least one of the following for the first task: task success rate, memory pool storage overhead, retrieval time, and tool call success rate.

[0108] In this embodiment, the task success rate can characterize whether the first task was successfully completed. The first task usually has clear completion criteria, such as whether the user's request was met or whether the expected goal was achieved. The task success rate can be directly obtained from the task execution results, and its value can be, for example, a binary success or failure indicator, or a continuous completion score.

[0109] Memory pool storage overhead characterizes the storage resources consumed by the memory pool when storing memory data. The more memories stored in the memory pool and the richer the content, the larger the required storage space typically is. Memory pool storage overhead reflects the extent to which the current memory management strategy utilizes storage resources.

[0110] Retrieval time characterizes the time spent from initiating a memory retrieval request to obtaining the retrieval result. Retrieval time is related to factors such as the size of the memory pool, the efficiency of the retrieval algorithm, and the organization of the memory data. A longer retrieval time may indicate that too much irrelevant or redundant memory has accumulated in the memory pool.

[0111] Tool invocation success rate characterizes the proportion of times the expected results are obtained when external tools are invoked during the execution of the first task. External tools can be, for example, search engines, databases, application programming interfaces (APIs), and code execution environments. Tool invocation success rate reflects whether the memory retrieval results provide effective support for tool invocation, such as whether they help determine the correct tool, parameters, or invocation method.

[0112] It should be noted that the task success rate, memory pool storage overhead, retrieval time, and tool call success rate corresponding to the first task mentioned above can be obtained individually or in any combination. This embodiment does not limit the specific method of obtaining these parameters.

[0113] Step S1022: Generate a reward value based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate.

[0114] In this embodiment, there are multiple ways to generate the reward value. For example, the acquired execution feedback metrics (i.e., the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate) can be weighted and combined according to preset weights to obtain a scalar reward value. The specific weighting method can be set according to actual needs, and this embodiment does not impose specific limitations on it.

[0115] As a quantitative form of execution feedback, the reward value reflects the overall performance of the first task under the first weight coefficient configuration, providing a basis for adjusting the weight coefficient. A high reward value indicates that the memory retrieval results under the current weight coefficient configuration have played a positive role in task execution; a low reward value indicates that the current weight coefficient configuration may be unreasonable and needs to be corrected in subsequent adjustments.

[0116] Accordingly, step S103 may include, but is not limited to: Step S1033: At least based on the state parameters, the reward value, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, adjust the first weight coefficients corresponding to each of the multiple attribute parameters to obtain the second weight coefficients.

[0117] In this embodiment, by acquiring at least one of the following parameters for the first task: task success rate, memory pool storage overhead, retrieval time, and tool invocation success rate, and generating a reward value based on the acquired information, a quantitative basis for adjusting the weight coefficients is provided. By quantifying the above multi-dimensional information into a unified reward value, the actual performance of the weight coefficient configuration can be comprehensively reflected from multiple perspectives, such as task completion, storage resource consumption, retrieval efficiency, and tool invocation effect. This provides a more comprehensive and accurate basis for adjusting the weight coefficients, thereby improving the targeting and effectiveness of the weight coefficient adjustment.

[0118] In one optional implementation, step S1022 may include, but is not limited to: Step S10221: Obtain the first reward weight corresponding to the task success rate, the second reward weight corresponding to the memory pool storage overhead, the third reward weight corresponding to the retrieval time, and the fourth reward weight corresponding to the tool call success rate.

[0119] When generating reward values ​​based on task success rate, memory pool storage overhead, retrieval time, and tool call success rate, different types of metrics may contribute to the reward value in different ways. For example, higher task success rate and tool call success rate indicate a better effect of the current weighting coefficient configuration, so increasing these two metrics should increase the reward value; while higher memory pool storage overhead and retrieval time indicate that the current memory management strategy consumes more storage resources and retrieval time, so increasing these two metrics should decrease the reward value.

[0120] To achieve the aforementioned differentiated contributions, this step assigns a corresponding reward weight to each metric. Specifically, task success rate corresponds to the first reward weight, memory pool storage overhead corresponds to the second reward weight, retrieval time corresponds to the third reward weight, and tool call success rate corresponds to the fourth reward weight. These reward weights are used to control the degree of contribution of each metric in the reward value calculation, that is, to determine the magnitude of the impact of changes in each metric on the reward value.

[0121] The values ​​of the aforementioned reward weights can be set according to the needs of the actual application scenario. For example, in scenarios with limited storage resources, a larger second reward weight can be set to increase the constraint on memory pool storage overhead and encourage the system to more actively clean up redundant memories; in scenarios with high requirements for response speed, a larger third reward weight can be set to strengthen the constraint on retrieval time and encourage the system to optimize retrieval efficiency; in scenarios where task execution quality is prioritized, larger first and fourth reward weights can be set to highlight the dominant position of task success rate and tool call success rate in the reward value. In addition, the values ​​of each reward weight can also be determined through empirical setting, experimental optimization, or online learning, and this implementation method does not impose specific restrictions on this.

[0122] Step S10222: Based on the first reward weight, the second reward weight, the third reward weight, and the fourth reward weight, process the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate respectively to obtain the reward value.

[0123] The first reward weight and the fourth reward weight are used to generate positive rewards for the task success rate and the tool call success rate, that is, the higher the task success rate and the tool call success rate, the greater the reward value.

[0124] The second reward weight and the third reward weight are used to generate negative rewards for the memory pool storage overhead and the retrieval time, that is, the greater the memory pool storage overhead and the retrieval time, the smaller the reward value.

[0125] Specifically, the reward value can be calculated using the following formula:

[0126] Where R represents the reward value, Indicates the task success rate. Indicates the memory pool storage overhead. Indicates the time taken for the retrieval. Indicates the success rate of tool invocation; Indicates the first reward weight. Indicates the weight of the second reward. Indicates the third reward weight. This indicates the fourth reward weight.

[0127] In the above relationship, task success rate and tool call success rate are indicators with the highest possible expectation. The higher the task success rate and tool call success rate, the better the support effect of memory retrieval results on task execution. Therefore, their corresponding reward weights are positive, so that the increase of these two indicators has a positive contribution to the reward value. Memory pool storage overhead and retrieval time are indicators with the lowest possible expectation. The higher the storage overhead and the longer the retrieval time, the worse the memory management strategy performs in terms of resource consumption and retrieval efficiency. Therefore, their corresponding reward weights are negative, so that the increase of these two indicators has a negative contribution to the reward value.

[0128] In this embodiment, the reward value is determined based on four dimensions of execution feedback indicators. This ensures that the reward value encompasses not only the direct effect of task execution success, but also indirect effects such as storage resource consumption and retrieval efficiency, as well as intermediate process information regarding the success of tool invocation. Compared to relying solely on the single indicator of task success, the reward value generated in this embodiment more comprehensively reflects the actual effect of the weight coefficient configuration, providing a more comprehensive basis for adjusting the weight coefficients.

[0129] In this embodiment, by configuring corresponding reward weights for task success rate, memory pool storage overhead, retrieval time, and tool call success rate, and generating reward values ​​based on each indicator and its corresponding reward weight, differentiated processing of different types of execution feedback information is achieved. Specifically, task success rate and tool call success rate are included in the reward value as positive rewards, incentivizing the system to improve task execution performance and tool call success rate. Memory pool storage overhead and retrieval time are included in the reward value as negative rewards, constraining the system to control storage resource consumption and retrieval time costs. Through this reward mechanism combining positive incentives and negative constraints, this implementation allows the adjustment of weight coefficients to balance resource consumption and efficiency while pursuing task execution performance, avoiding focusing solely on task success rate while neglecting storage and retrieval costs, thereby guiding the adjustment of weight coefficients towards a more balanced and efficient direction.

[0130] As another optional embodiment of this application, refer to Figure 4 This is a flowchart illustrating a processing method provided in Embodiment 4 of this application. This embodiment is mainly an implementation of step S102 in Embodiment 1, such as... Figure 4 As shown, the specific steps may include, but are not limited to, the following: Step S1023: Obtain the immediate reward, process reward, and final reward corresponding to the first task.

[0131] The instant reward is determined based on at least one of the following: the adoption rate of the memory retrieval results corresponding to the first task, the redundancy of the retrieval results, and the relevance of the retrieval results. The process reward is determined based on at least one of the tool call success rate corresponding to the first task, repeated call penalty, and failure experience avoidance reward. The final state reward is determined based on the task success rate and memory efficiency reward corresponding to the first task.

[0132] In Example 3, the execution feedback information is refined into metrics such as task success rate, memory pool storage overhead, retrieval time, and tool call success rate, and a unified reward value is generated through a weighted average. However, this reward value is only obtained once upon task completion, representing a final state signal. In practical applications, the execution process of the first task typically involves multiple steps, such as multiple memory retrievals and multiple tool calls. Relying solely on the final state signal results in a sparse reward signal; that is, the system can only obtain feedback once at the end of the task and cannot know the performance of each intermediate step during execution, making it difficult for the reinforcement learning strategy to converge effectively.

[0133] Based on this, this embodiment obtains immediate rewards, process rewards, and final state rewards, which correspond to the performance at different stages during task execution.

[0134] Immediate reward can be understood as a reward signal triggered each time a memory retrieval or write operation occurs. Immediate reward is used to evaluate the quality of a single memory retrieval or write operation, aiming to guide the system to make high-quality choices in each retrieval or write operation. Immediate reward can be calculated using the following formula: R_immediate=β1×HitRate+β2×(1-Redundancy)+β3×RetrievalPrecision Here, R_immediate represents the immediate reward; HitRate represents the memory retrieval result adoption rate, which is the proportion of retrieved memories actually used by the task planner. A higher adoption rate indicates that the retrieval results are more closely aligned with the task requirements; Redundancy represents the retrieval result redundancy, which is the degree of overlap between the current retrieval result and the previous retrieval result. Excessive redundancy indicates a lack of diversity in the retrieval results, which may lead to the system repeatedly obtaining the same information while ignoring other useful information; RetrievalPrecision represents the retrieval result relevance, which is the average semantic relevance between the retrieval result and the current task. A higher relevance indicates a better match between the retrieval result and the current task; β1, β2, and β3 represent the weight coefficients corresponding to the above three indicators, respectively.

[0135] Process rewards can be understood as reward signals triggered after each tool invocation. Process rewards are used to evaluate the quality of a single tool invocation, aiming to guide the system to fully utilize historical experience and avoid repeating mistakes when invoking tools. Process rewards can be calculated using the following formula: R_process = γ1×ToolSuccessScore - γ2×RepeatCallPenalty + γ3×FailureAvoidanceBonus Wherein, R_process represents the process reward; ToolSuccessScore represents the tool call success rate, i.e. whether the tool call successfully obtained the expected result; RepeatCallPenalty represents the repeated call penalty, i.e. whether the tool and parameters of the current call are the same as those of the previous failed call. If they are the same, a penalty is applied to prevent the system from repeatedly trying the failed call method; FailureAvoidanceBonus represents the failure experience avoidance reward, i.e. if there is a historical failure record of the tool or parameters in the memory pool, and the record is successfully retrieved and adopted by the system to avoid similar errors, a positive reward is given to incentivize the system to actively retrieve and utilize historical failure experience; γ1, γ2, and γ3 represent the weight coefficients corresponding to the above three indicators, respectively.

[0136] Final state reward can be understood as the reward signal triggered when the first task is completed. It is used to evaluate the overall performance of the task and aims to guide the system to optimize its memory management strategy from a global perspective. Final state reward can be calculated using the following formula: R_terminal = TaskSuccessRate × (1 + MemoryEfficiencyBonus) Here, R_terminal represents the final state reward; TaskSuccessRate represents the task success rate, i.e., whether the first task was successfully completed; MemoryEfficiencyBonus represents the memory efficiency reward, calculated as RetainedMemoryValue divided by TotalMemorySlots, where RetainedMemoryValue represents the total effective value of the memories related to the first task retained after its completion, and TotalMemorySlots represents the total number of available slots in the memory pool. MemoryEfficiencyBonus can be used to measure the efficiency of memory slot utilization when completing tasks of equal quality. It provides an extra reward for completing tasks of equal quality using fewer memory slots, thus incentivizing the system to learn a streamlined memory strategy rather than indiscriminately retaining all memories.

[0137] Step S1024: Weight the immediate reward, the process reward, and the final state reward to obtain the total reward value.

[0138] The total reward value can be calculated using the following formula: R_total = λ1×R_immediate +λ2×R_process +λ3×R_terminal Where R_total represents the total reward value; R_immediate represents the immediate reward; R_process represents the process reward; R_terminal represents the final reward; and λ1, λ2, and λ3 represent the weight coefficients corresponding to the immediate reward, process reward, and final reward, respectively.

[0139] The total reward value, as a quantitative representation of execution feedback, reflects the performance of the first task at each stage of execution. Specifically, the immediate reward provides fine-grained feedback during memory retrieval and writing, the process reward provides intermediate feedback during tool invocation, and the final state reward provides an overall evaluation upon task completion. The total reward value, obtained by weighting and summing these three values, is more densely packed in the time dimension than a single final state signal, effectively overcoming the reward sparsity problem caused by relying solely on a single feedback signal at task completion. Each memory retrieval, each tool invocation, and the final task completion during task execution all provide feedback information to the total reward value, making the execution feedback information more comprehensive and facilitating a more complete basis for adjusting the weighting coefficients.

[0140] Accordingly, step S103 may include, but is not limited to: Step S1034: Adjust the first weight coefficients corresponding to each of the multiple attribute parameters based at least on the state parameters, the total reward value, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters to obtain the second weight coefficients.

[0141] After obtaining the total reward value, this step takes the state parameters, the total reward value, and the attribute values ​​of each first candidate stored on multiple attribute parameters as input to adjust the first weighting coefficient to obtain the second weighting coefficient. The difference from step S103 in Example 1 is that the execution feedback information used in this step is the total reward value obtained by weighting the immediate reward, process reward, and final state reward, rather than a single final state signal.

[0142] Because the total reward value incorporates feedback information from multiple stages during task execution, this step allows for a more nuanced differentiation of the impact of performance differences at different stages on weight coefficient adjustments, compared to solutions that rely solely on final state signals. For example, if the immediate reward is low but the final state reward is high, it indicates a potential deficiency in the memory retrieval stage, but the overall task is still successfully completed; in this case, the weight coefficient adjustment can be relatively small. Conversely, if both the process reward and the final state reward are low, it suggests a problem with the tool invocation stage and overall task failure; in this case, the weight coefficient adjustment needs to be relatively large. Through this tiered reward mechanism, weight coefficient adjustments can be based on richer feedback signals, thereby improving the accuracy of the adjustments.

[0143] As another optional embodiment of this application, a processing method provided in Embodiment 5 of this application is mainly an implementation of obtaining the status parameters corresponding to the current task in Embodiment 1. Specifically, it may include, but is not limited to, at least one of the following: Step S1011: Obtain the type of the first task as the status parameter corresponding to the execution of the first task.

[0144] Different task types have different memory requirements; therefore, the type of the first task can provide a directional reference for adjusting the weighting coefficients.

[0145] The type of the first task can be identified by analyzing information such as the task content, user request method, and task objectives. For example, the first task can be classified based on pre-defined task classification rules or a trained task classification model to obtain the corresponding task type.

[0146] The types of the first task may include, but are not limited to, the following: Question-answering tasks. Users pose questions, and the agent needs to answer based on historical dialogues, knowledge bases, or tool results. The core of this type of task lies in accurately acquiring information relevant to the question. Therefore, the weights of task relevance (i.e., the second attribute parameter) and information confidence (i.e., the third attribute parameter) should be relatively high to ensure that the retrieved memories are both highly relevant to the question and from reliable sources.

[0147] Multi-turn dialogue tasks require agents to continuously understand the context, such as continuously modifying solutions, tracking user preferences, and following up on historical discussions. These tasks demand high coherence of the context; remembering recent dialogue records, user preference information, and historical modification records is particularly important. Therefore, the weights of the first attribute parameter reflecting storage duration (for retaining recent context) and the fourth attribute parameter reflecting access frequency (for identifying high-frequency user preferences) should be appropriately increased.

[0148] Planning tasks. Agents need to complete complex goals step-by-step, such as planning itineraries, generating project plans, and breaking down tasks. These tasks involve multiple stages of decision-making and state tracking; the memory of mid-term task states, stage-by-stage decision records, and unfinished steps is crucial for task progress. Therefore, the weight of the fifth attribute parameter (preventing key planning steps from being forgotten), which reflects the risk of forgetting, should be relatively high.

[0149] Tool invocation tasks. The agent needs to invoke external tools such as search engines, databases, APIs, and code execution. These tasks heavily rely on the tool invocation history, including memories of successful tool invocation paths, failure reasons, and interface parameters. Therefore, the weights of the fourth attribute parameter (reflecting access frequency to identify frequently used tools and successful paths) and the fifth attribute parameter (reflecting the risk of forgetting (preventing lessons learned from failures) should be relatively high.

[0150] Content generation tasks, such as generating reports, emails, patent texts, and abstracts, rely heavily on user writing preferences, formatting requirements, target audience, and feedback from previous versions. Therefore, the weights of the second attribute parameter (reflecting semantic similarity to match user-preferred content styles) and the third attribute parameter (reflecting the reliability of data sources to ensure the credibility of cited information) should be relatively high.

[0151] Risk control tasks, such as those involving security, compliance, privacy, and sensitive operational judgments, are highly dependent on memories of high-risk events, error feedback, and compliance rules. Missing crucial risk memories could have serious consequences. Therefore, the fifth attribute parameter, reflecting the risk of forgetting, should be given the highest weight to ensure that memories containing security rules and lessons learned are prioritized for retrieval and retention.

[0152] Step S1012: Determine the occupancy rate of the memory pool as a status parameter corresponding to the execution of the first task.

[0153] When the memory pool occupancy rate is low, there is still sufficient storage space in the memory pool to accommodate new memories. At this time, the pressure to eliminate memories is relatively small, and the adjustment of weight coefficients can focus more on improving the accuracy of memory retrieval. For example, the weights of semantic similarity (second attribute parameter) and information confidence (third attribute parameter) can be increased to prioritize the quality of retrieval.

[0154] When the memory pool is occupancy rate is high, the memory pool faces pressure from storage space shortage. At this time, it is necessary to make a more careful decision on which memories should be retained and which memories can be eliminated. Therefore, the weights of the fifth attribute parameter reflecting the risk of forgetting and the first attribute parameter reflecting the storage duration should be increased accordingly to ensure that high-value memories are not eliminated prematurely, while encouraging the timely clearing of expired or useless memories.

[0155] In this embodiment, the type of the first task and the occupancy rate of the memory pool can be used as state parameters individually or jointly. When both are used as state parameters, the objective scenario after the first task is completed can be reflected more comprehensively. The task type provides information on the task demand side, while the occupancy rate provides information on the resource supply side. During the adjustment of the weighting coefficients, both, along with the execution feedback information and the attribute values ​​of the first candidate memory on multiple attribute parameters, are used as inputs. This allows the adjustment of the weighting coefficients to simultaneously consider task demand, resource pressure, execution effect, and the actual status of the memory, thereby improving the accuracy and effectiveness of the adjustment.

[0156] In one alternative implementation, determining the occupancy rate of the memory pool may include any of the following: Step S10121: Determine the ratio of the used storage space of the memory pool to the total storage space, as the occupancy rate of the memory pool.

[0157] Used storage space can be understood as the total storage space occupied by all currently stored memories in the memory pool. When each memory is stored in the memory pool, its data content will occupy a certain amount of storage resources. The used storage space of each memory can be accumulated to obtain the used storage space of the memory pool.

[0158] Total storage space can be understood as the maximum storage space allocated to the memory pool that can be used to store memories. By dividing the used storage space by the total storage space, we can obtain a value between 0 and 1, which is the memory pool occupancy rate determined based on the storage space dimension.

[0159] This method measures the load of the memory pool from the perspective of storage resource usage. When the data size of each memory in the memory pool varies greatly, for example, some memories are short texts and some memories are longer documents or image descriptions, the occupancy rate calculated based on storage space can more accurately reflect the actual usage of storage resources by the memory pool, avoiding the bias caused by the difference in memory size when measuring only by the number of memory entries.

[0160] Step S10122: Determine the ratio of the number of memories already stored in the memory pool to the maximum number of memories that can be stored, as the occupancy rate of the memory pool.

[0161] The number of stored memories can be understood as the total number of memory entries currently stored in the memory pool. Each memory is treated as an independent storage unit, counted as 1. By counting the total number of memory entries in the current memory pool, the number of stored memories can be obtained. The maximum storable number can be understood as the upper limit of the number of memory entries that the memory pool is allowed to store. By dividing the number of stored memories by the maximum storable number, a value between 0 and 1 can be obtained, which is the memory pool occupancy rate determined based on the number of memory entries.

[0162] This method measures the load of the memory pool from the perspective of the number of memory entries. When the data size of each memory in the memory pool is relatively uniform, or when the storage management of the memory pool is controlled by the number of entries, the occupancy rate calculated based on the number of memories can simply and intuitively reflect the fullness of the memory pool, and the calculation method is relatively simple.

[0163] In actual implementation, the memory pool occupancy rate can be determined by any of the above methods based on the memory pool's storage management mechanism and the actual characteristics of the memory data. Alternatively, a combination of the two methods can be used for comprehensive judgment. This embodiment does not impose any specific restrictions on this.

[0164] As another optional embodiment of this application, refer to Figure 5 This is a flowchart illustrating a processing method provided in Embodiment 6 of this application, as shown below. Figure 5 As shown, the method may include, but is not limited to, the following steps: Step S201: Obtain the status parameters corresponding to the execution of the first task, and the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters.

[0165] Step S202: Obtain the execution feedback information of the first task.

[0166] Step S203: Based at least on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, adjust the first weight coefficients corresponding to each of the multiple attribute parameters to obtain the second weight coefficients.

[0167] For a detailed description of steps S201-S203, please refer to the relevant description of steps S101-S103 in Example 1, which will not be repeated here.

[0168] Step S204: At least based on the state parameters, the execution feedback information, and the attribute values ​​of each first candidate memory on multiple attribute parameters, perform an update operation on a portion of the first candidate memories in the memory pool to obtain second candidate memories.

[0169] In this embodiment, the update operation may include, but is not limited to, at least one of the following: retain, discard, compress, and promote operations.

[0170] After the first task is completed, the memory pool contains both memories that existed before the current task and memories newly generated during the task. These memories collectively constitute the first candidate memories. However, not all first candidate memories are worth retaining. Some memories may be expired, contain duplicate content, have a weak long-term connection to the task, or occupy too much storage space but have low value. If these memories are not processed, the storage pressure on the memory pool will continue to increase, retrieval efficiency will gradually decrease, and irrelevant memories may interfere with memory selection in subsequent tasks.

[0171] Therefore, this step performs at least one of the following operations—retention, discarding, and compression—on a portion of the first candidate memories in the memory pool, based on the state parameters, execution feedback information, and the attribute values ​​of the first candidate memories on multiple attribute parameters.

[0172] In one alternative implementation, the decision of which memories to retain, discard, or compress can be determined by the reinforcement learning scheduler. Specifically, after each task execution, the reinforcement learning scheduler outputs a corresponding action based on state parameters, execution feedback information, and the attribute values ​​of the first candidate memories across multiple attribute parameters. This action may include two parallel parts: the first part is the adjustment of weight coefficients as described in step S203, and the second part is the retention, discarding, or compression operation on a portion of the first candidate memories in the memory pool as described in this step.

[0173] For example, when the memory pool is occupancy is high, the reinforcement learning scheduler can output a discard or compress operation to free up storage space; when the execution feedback indicates that the task failed due to a lack of key lessons learned, the scheduler can output a retain operation to ensure that memories at high risk of being forgotten are not discarded.

[0174] The retention operation can be understood as maintaining the storage state of the memory in the memory pool. High-value memories, memories containing key lessons learned from failures, and memories recording important user preferences should be retained to ensure that these memories can still be retrieved in subsequent tasks.

[0175] The discard operation can be understood as deleting the memory from the memory pool, freeing up the storage space it occupies. Discard operations can be performed on memories that are expired, contain duplicate content, have weak relevance to all task types, and are no longer of reference value.

[0176] Compression can be understood as reducing the stored content of a memory without completely deleting it. For example, long memory texts can be summarized, retaining the core information while removing detailed descriptions; or multiple similar memories can be merged into one, reducing the number of memory entries. Compression is suitable for memories that have some reference value but high storage costs, reducing storage costs while retaining useful information.

[0177] Boosting can be understood as increasing the priority of memories containing high-risk events or lessons learned from failures in subsequent retrievals without directly modifying the stored content of the memory. In one alternative implementation, boosting can be achieved by increasing the forgetting risk attribute value of the memory, thus giving it a higher overall score when calculating attribute factors based on a second weighting coefficient. In another alternative implementation, boosting can be achieved by attaching a priority marker to the memory, prioritizing memories with this marker in subsequent retrievals. Boosting is suitable for memories with a high forgetting risk that may be easily overlooked in subsequent retrievals due to lower values ​​in other attributes, especially memories containing high-risk events or lessons learned from failures.

[0178] Step S205: Based on the second weighting coefficient, process the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters, and determine the attribute factors of each second candidate memory.

[0179] For a detailed description of step S205, please refer to the relevant description of step S104 in Example 1, which will not be repeated here.

[0180] In this embodiment, the retention operation ensures that high-value memories can continue to exist in the memory pool, avoiding the risk of losing critical memories due to blind clearing.

[0181] By discarding data, low-value, outdated, or redundant memories are promptly cleared, reducing the occupancy rate of the memory pool and minimizing interference during retrieval, thereby shortening retrieval time and improving the efficiency of memory retrieval.

[0182] By performing compression, storage space consumption is reduced while preserving core memory information, achieving a balance between memory retention and resource consumption.

[0183] By improving the operation, memories containing high-risk events or lessons learned from failures can be given higher priority in subsequent retrievals. This avoids the risk of such memories being ignored by subsequent retrievals because they have low values ​​on certain attribute dimensions. As a result, the memory management strategy is enhanced to protect high-risk and failure experience memories, and the agent's ability to avoid similar errors is improved.

[0184] Furthermore, the execution basis for retention, discarding, compression, and promotion operations is consistent with the adjustment basis for weight coefficients. Both are based on state parameters, execution feedback information, and the attribute values ​​of the first candidate memory. This ensures that the physical cleaning of the memory pool and the logical adjustment of the weight coefficients are coordinated and work together to improve the accuracy of memory retrieval and the performance of task execution in subsequent tasks.

[0185] In this embodiment, combined with Figure 6 The processing method will be explained. For example... Figure 6 As shown, the overall process of this method is as follows: The input to the reinforcement learning scheduler includes the following three types of information: The first type of information comes from the task feedback module, including status parameters and reward R. The status parameters describe the objective scenario after the first task is completed, including the type of the first task and the memory pool occupancy rate. The reward R quantifies the execution feedback information of the first task, such as a reward value generated based on task success rate, memory pool storage overhead, retrieval time, and tool call success rate, or a total reward value obtained by weighting immediate rewards, process rewards, and final state rewards.

[0186] The second type of information comes from the weight calculator, including the attribute factors of the first candidate memory and the first weight coefficient α. The attribute factors of the first candidate memory are obtained by processing the attribute values ​​of the first candidate memory across multiple attribute parameters based on the first weight coefficient; the first weight coefficient α is the weight coefficient corresponding to each attribute parameter currently in use.

[0187] The third type of information comes from the memory pool manager, including the attribute values ​​of the first candidate memory across multiple attribute parameters. These attribute values ​​reflect the actual values ​​of each memory in the memory pool in dimensions such as storage duration t, semantic similarity I, data source reliability C, access frequency F, and forgetting risk R.

[0188] The reinforcement learning scheduler outputs action A based on the three types of input information mentioned above. Action A consists of two parts: the first part is the adjustment instruction for the first weight coefficient α, i.e., the adjusted second weight coefficient; the second part is the management operation instruction for a portion of the first candidate memories in the memory pool, including at least one of the operations of retention, discarding, compression, and promotion.

[0189] The action executor receives action A from the reinforcement learning scheduler and performs two operations: First, according to the weight coefficient adjustment instruction, it updates the first weight coefficient to the second weight coefficient and sends the updated second weight coefficient to the weight calculator; second, according to the retain, discard, compress, and promote operation instructions, it performs corresponding management operations on the first candidate memory in the memory pool and updates the memory pool. The updated memory pool contains the second candidate memory obtained after the retain, discard, compress, and promote operations.

[0190] When a user initiates a request to execute a second task, the Planning and Controller responds by sending a request to the Memory Pool Manager to retrieve the context. Upon receiving the request, the Memory Pool Manager sends a request to the Weight Calculator to retrieve the weight coefficients. The Weight Calculator calculates the attribute factors of each candidate second memory based on the updated second weight coefficients from the action executor and returns the results to the Memory Pool Manager. The Memory Pool Manager selects the memory whose score meets preset conditions as the target memory based on the attribute factors of the candidate second memories and returns it to the Planning and Controller for executing the second task.

[0191] After acquiring the target memory, the planning and controller makes a tool invocation decision and sends a tool invocation request to the tool layer. The tool layer includes various external tools such as Web Search, RAG Knowledge Base (RAG KB), and Enterprise Plugins. The tool layer executes the corresponding tool operation based on the invocation request and returns the invocation result to the planning and controller.

[0192] The planning and control system sends the tool call results and the execution results of the second task to the task feedback module. Based on the received information, the task feedback module generates state parameters and a reward R, which serve as input to the reinforcement learning scheduler after the next task execution, thus forming a continuous closed-loop iterative optimization process.

[0193] Through the above process, this embodiment achieves coordinated optimization of dynamic adjustment of weight coefficients and memory pool management. The reinforcement learning scheduler, based on multi-source information such as task feedback, weight calculation, and memory pool status, makes unified decisions on the direction of weight adjustment and memory management operations. This ensures that the logical adjustment of weight coefficients and the physical clearing of the memory pool work together to improve the accuracy of memory retrieval and the performance of task execution in subsequent tasks.

[0194] The processing apparatus provided in this application will be described below. The processing apparatus described below can be referred to in correspondence with the processing method described above.

[0195] The processing device may include: The first acquisition module is used to acquire the status parameters corresponding to the execution of the first task, as well as the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters.

[0196] The second acquisition module is used to acquire the execution feedback information of the first task.

[0197] The adjustment module is used to adjust the first weight coefficients corresponding to the multiple attribute parameters based at least on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, to obtain the second weight coefficients.

[0198] The first processing module is used to process the attribute values ​​of each second candidate memory in the memory pool on the plurality of attribute parameters based on the second weighting coefficient, and to determine the attribute factors of each second candidate memory.

[0199] In this embodiment, the adjustment module can be specifically used for: Obtain the attribute factors of each of the first candidate memories; the attribute factors of the first candidate memories are obtained by processing the attribute values ​​of the first candidate memories on multiple attribute parameters based on the first weight coefficient. Based on the state parameters, the execution feedback information, the attribute values ​​of each first candidate memory on multiple attribute parameters, and the attribute factors of each first candidate memory, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients.

[0200] In this embodiment, the second acquisition module can be specifically used for: Obtain at least one of the following for the first task: task success rate, memory pool storage overhead, retrieval time, and tool call success rate; A reward value is generated based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate.

[0201] The second acquisition module generates a reward value based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate, which may specifically include: Obtain the first reward weight corresponding to the task success rate, the second reward weight corresponding to the memory pool storage overhead, the third reward weight corresponding to the retrieval time, and the fourth reward weight corresponding to the tool call success rate; The reward value is obtained by processing the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate according to the first reward weight, the second reward weight, the third reward weight, and the fourth reward weight respectively. The first reward weight and the fourth reward weight are used to generate positive rewards for the task success rate and the tool call success rate, while the second reward weight and the third reward weight are used to generate negative rewards for the memory pool storage overhead and the retrieval time.

[0202] In this embodiment, the second acquisition module can be specifically used for: Obtain the immediate reward, process reward, and final state reward corresponding to the first task; The immediate reward, the process reward, and the final state reward are weighted to obtain the total reward value. The instant reward is determined based on at least one of the following: the adoption rate of the memory retrieval results corresponding to the first task, the redundancy of the retrieval results, and the relevance of the retrieval results. The process reward is determined based on at least one of the tool call success rate corresponding to the first task, repeated call penalty, and failure experience avoidance reward. The final state reward is determined based on the task success rate and memory efficiency reward corresponding to the first task.

[0203] In this embodiment, the first acquisition module acquires the status parameters corresponding to the execution of the first task, which may include at least one of the following: Obtain the type of the first task, which will be used as the status parameter for executing the first task; The occupancy rate of the memory pool is determined and used as a status parameter for executing the first task.

[0204] In this embodiment, the first acquisition module determines the occupancy rate of the memory pool, which may include any one of the following: The ratio of the used storage space to the total storage space of the memory pool is determined as the occupancy rate of the memory pool; The ratio of the number of memories already stored in the memory pool to the maximum number that can be stored is determined as the occupancy rate of the memory pool.

[0205] In this embodiment, the processing device may further include: The second processing module is used to perform an update operation on a portion of the first candidate memories in the memory pool based at least on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on multiple attribute parameters, to obtain the second candidate memories.

[0206] In this embodiment, the multiple attribute parameters may include any of the following: The first attribute parameter is used to reflect the storage duration of the corresponding candidate memory in the memory pool; The second attribute parameter is used to reflect the semantic similarity between the corresponding candidate memory and the corresponding task in the memory pool; The third attribute parameter is used to reflect the reliability of the data source of the corresponding candidate memory in the memory pool; The fourth attribute parameter is used to reflect the frequency with which the corresponding candidate memory in the memory pool is called in historical tasks; The fifth attribute parameter is used to reflect the risk level of the corresponding task failing to execute after the corresponding candidate memory in the memory pool is lost.

[0207] In another embodiment of this application, an electronic device is provided, comprising: processor; Memory; The memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the following method steps: Obtain the status parameters corresponding to the execution of the first task, as well as the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters; Obtain the execution feedback information of the first task; At least based on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients; Based on the second weighting coefficient, the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters are processed to determine the attribute factors of each second candidate memory.

[0208] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.

[0209] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CPUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, training equipment, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0210] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.

[0211] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, training device, or data center to another website, computer, training device, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a training device or data center that integrates one or more available media. The available media may be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

Claims

1. A processing method, comprising: Obtain the status parameters corresponding to the execution of the first task, as well as the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters; Obtain the execution feedback information of the first task; At least based on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients; Based on the second weighting coefficient, the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters are processed to determine the attribute factors of each second candidate memory.

2. The processing method according to claim 1, wherein adjusting the first weight coefficients corresponding to each of the plurality of attribute parameters based at least on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the plurality of attribute parameters to obtain the second weight coefficients, includes: Obtain the attribute factors of each of the first candidate memories; The attribute factor of the first candidate memory is obtained by processing the attribute values ​​of the first candidate memory on multiple attribute parameters based on the first weight coefficient. Based on the state parameters, the execution feedback information, the attribute values ​​of each first candidate memory on multiple attribute parameters, and the attribute factors of each first candidate memory, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients.

3. The processing method according to claim 1, wherein obtaining the execution feedback information of the first task includes: Obtain at least one of the following for the first task: task success rate, memory pool storage overhead, retrieval time, and tool call success rate; A reward value is generated based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate.

4. The processing method according to claim 3, wherein generating a reward value based on at least one of the task success rate, the memory pool storage overhead, the retrieval time, and the tool invocation success rate includes: Obtain the first reward weight corresponding to the task success rate, the second reward weight corresponding to the memory pool storage overhead, the third reward weight corresponding to the retrieval time, and the fourth reward weight corresponding to the tool call success rate; The reward value is obtained by processing the task success rate, the memory pool storage overhead, the retrieval time, and the tool call success rate according to the first reward weight, the second reward weight, the third reward weight, and the fourth reward weight respectively. The first reward weight and the fourth reward weight are used to generate positive rewards for the task success rate and the tool call success rate, while the second reward weight and the third reward weight are used to generate negative rewards for the memory pool storage overhead and the retrieval time.

5. The processing method according to claim 1, wherein, The step of obtaining the execution feedback information of the first task includes: Obtain the immediate reward, process reward, and final state reward corresponding to the first task; The immediate reward, the process reward, and the final state reward are weighted to obtain the total reward value. The instant reward is determined based on at least one of the following: the adoption rate of the memory retrieval results corresponding to the first task, the redundancy of the retrieval results, and the relevance of the retrieval results. The process reward is determined based on at least one of the tool call success rate, repeated call penalty, and failure experience avoidance reward corresponding to the first task; The final state reward is determined based on the task success rate and memory efficiency reward corresponding to the first task.

6. The processing method according to claim 1, wherein obtaining the status parameters corresponding to the execution of the first task includes at least one of the following: Obtain the type of the first task, which will be used as the status parameter for executing the first task; The occupancy rate of the memory pool is determined and used as a status parameter for executing the first task.

7. The processing method according to claim 6, wherein determining the occupancy rate of the memory pool includes any one of the following: The ratio of the used storage space to the total storage space of the memory pool is determined as the occupancy rate of the memory pool; The ratio of the number of memories already stored in the memory pool to the maximum number that can be stored is determined as the occupancy rate of the memory pool.

8. The processing method according to claim 1, further comprising: At least based on the state parameters, the execution feedback information, and the attribute values ​​of each first candidate memory on multiple attribute parameters, an update operation is performed on a portion of the first candidate memories in the memory pool to obtain second candidate memories.

9. The processing method according to claim 1, wherein the plurality of attribute parameters includes any one of the following: The first attribute parameter is used to reflect the storage duration of the corresponding candidate memory in the memory pool; The second attribute parameter is used to reflect the semantic similarity between the corresponding candidate memory and the corresponding task in the memory pool; The third attribute parameter is used to reflect the reliability of the data source of the corresponding candidate memory in the memory pool; The fourth attribute parameter is used to reflect the frequency with which the corresponding candidate memory in the memory pool is called in historical tasks; The fifth attribute parameter is used to reflect the risk level of the corresponding task failing to execute after the corresponding candidate memory in the memory pool is lost.

10. An electronic device, comprising: processor; Memory; The memory stores a computer program, which, when executed by the processor, causes the electronic device to perform the following method steps: Obtain the status parameters corresponding to the execution of the first task, as well as the attribute values ​​of each first candidate memory in the memory pool on multiple attribute parameters; Obtain the execution feedback information of the first task; At least based on the state parameters, the execution feedback information, and the attribute values ​​of each of the first candidate memories on the multiple attribute parameters, the first weight coefficients corresponding to each of the multiple attribute parameters are adjusted to obtain the second weight coefficients; Based on the second weighting coefficient, the attribute values ​​of each second candidate memory in the memory pool on the multiple attribute parameters are processed to determine the attribute factors of each second candidate memory.

Images