Implementation method and system of self-evolution learning of intelligent agent
By employing environmental feedback trial-and-error optimization strategies and transfer learning, the agent self-evolutionary learning method addresses the problem of insufficient task-solving capabilities of agents in complex and dynamic scenarios, enabling autonomous adaptation and continuous improvement.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-31
- Publication Date
- 2026-05-29
AI Technical Summary
Existing intelligent agents rely on human intervention when facing complex and dynamic scenarios, resulting in insufficient task-solving capabilities and a lack of versatility.
By employing an environmental feedback trial-and-error optimization strategy, and combining a proximal strategy optimization algorithm and a transfer learning optimization algorithm with a long short-term memory network and a knowledge graph, the agent can achieve self-evolutionary learning and autonomously adapt to complex dynamic scenarios.
Intelligent agents can continuously absorb new knowledge throughout their lifecycle, improve their task-solving abilities and generalization capabilities, reduce reliance on human intervention, and adapt to diverse environments.
Smart Images
Figure CN122114043A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the fields of agent reinforcement learning, large language models, memory and cognitive intelligence, and transfer learning, specifically to a method and system for implementing agent self-evolutionary learning. Background Technology
[0002] Large language model technology: provides core reasoning and perception capabilities, and is the brain foundation for the evolution of intelligent agents.
[0003] Reinforcement learning techniques, such as temporal difference learning and policy gradient methods, provide agents with a core learning logic of trial-and-error, feedback-and optimization, and are the core driving force for self-evolution.
[0004] Memory and storage technologies: Long Short-Term Memory (LSTM) networks, vector databases, and knowledge graphs support agents in storing experience, extracting key information, and enabling experience reuse and accumulation.
[0005] Learning framework technologies: lifelong learning frameworks and transfer learning algorithms solve the problems of continuous learning without forgetting and cross-task ability transfer, thus facilitating long-term evolution.
[0006] Optimization and training techniques: Adaptive learning rate scheduling, distributed training framework, and regularization methods ensure stable model iteration during the agent's evolution, preventing overfitting or degradation.
[0007] Existing intelligent agents rely on human intervention and have poor ability to adapt to complex and dynamic scenarios, resulting in insufficient task-solving capabilities and limited versatility. Summary of the Invention
[0008] The purpose of this invention is to provide a method and system for realizing the self-evolutionary learning of intelligent agents. Through environmental feedback and trial-and-error optimization strategies, the intelligent agent continuously absorbs new knowledge and achieves iterative evolution throughout its life cycle, enabling the intelligent agent to break free from dependence on human intervention, autonomously adapt to complex dynamic scenarios, and continuously improve its task-solving ability and generalization.
[0009] To address the aforementioned technical problems, this invention provides a method for implementing self-evolutionary learning of an intelligent agent, comprising the following steps: Obtain the environment status; The environmental state is input into the pre-trained policy, which then outputs the optimal action and executes it. After the action is performed, a feedback signal is acquired; the feedback signal includes environmental response and task completion status. The strategy is optimized based on the feedback signals to obtain a new strategy.
[0010] Preferably, the training method for the strategy employs a proximal policy optimization algorithm; The near-end strategy optimization algorithm specifically includes the following steps: Obtain historical data; Based on historical data, calculate the cumulative return of the discount and standardize it as the learning objective; based on the learning objective, obtain the Actor loss and Critic loss; The probability distribution of new actions is output through the network, and the probability ratio with the old strategy is calculated. The total loss is then optimized by combining the Actor loss and Critic loss. Repeat the iteration until the policy converges, and output the optimal policy as the pre-trained policy.
[0011] Preferably, the strategy is optimized using a transfer learning optimization algorithm; The transfer learning optimization algorithm includes: Load the pre-trained backbone model and freeze all its parameters.
[0012] Add a LoRA low-rank adaptation layer after the backbone model, and train only the parameters of this layer.
[0013] The features / parameters learned from the source task are transferred to the target task, and the feature output is fine-tuned through the LoRA low-rank adaptation layer to adapt to the requirements of the new task. By combining the target task data with the training of the classification head, the ability transfer from the source task to the target task is completed.
[0014] Preferably, the method further includes the following steps: Based on the environmental state, actions, feedback signals, environmental state after executing action a, and knowledge rules, vectors are transformed to generate long-term memory features. By structuring the characteristics of long-term memory, structured knowledge can be obtained. Based on the current task requirements, long-term memory features are extracted from structured knowledge as historical data.
[0015] Preferably, the long-term memory feature employs a long-term memory algorithm: Define the input dimension, hidden layer dimension, and number of LSTM layers to construct an LSTM network.
[0016] The vectors are fed into the LSTM network, which outputs the feature vectors at all time steps as long-term memory representations.
[0017] Preferably, the strategy is optimized based on the feedback signal to obtain a new strategy, specifically including the following steps: Obtain feedback signals; The feedback signal is normalized to obtain the normalized result; The normalized results are weighted and fused according to preset weights to obtain a comprehensive reward signal; Use the comprehensive reward signal as an optimizable loss value; The strategy is optimized based on the optimizable loss value.
[0018] Preferably, after obtaining the environmental status, the following steps are also included: The environment state is preprocessed through regularization and data augmentation.
[0019] Preferably, the method further includes the following steps: Protect historical task parameters during new task learning; Performance is validated using a test set after each evolution round.
[0020] This invention also provides a system for implementing self-evolutionary learning of an intelligent agent, comprising: The acquisition module is used to acquire environmental status. The execution module is used to input the environmental state into the pre-trained policy, output the optimal action, and execute it. The feedback module is used to acquire feedback signals after an action is performed; the feedback signals include environmental response and task completion status. The optimization module is used to optimize the strategy based on feedback signals to obtain a new strategy.
[0021] Compared with the prior art, the beneficial effects of the present invention are as follows: Continuous improvement in task performance: It can continuously optimize when faced with long-term or repetitive tasks, and become more and more proficient in handling complex cross-platform tasks.
[0022] Reduce R&D and application costs: Reduce reliance on manual labor, autonomously discover reinforcement learning rules, eliminate the need for continuous manual data labeling, and rely on environmental feedback for iterative optimization.
[0023] Expanding the boundaries of technology application: enabling concurrent interaction in diverse environments and adapting to tasks of varying difficulty, allowing intelligent agents to gradually acquire the general ability to solve complex tasks across environments. Attached Figure Description
[0024] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0025] Figure 1 This is a flowchart illustrating the implementation method of a self-evolutionary learning method for an intelligent agent according to the present invention. Detailed Implementation
[0026] Numerous specific details are set forth in the following description to provide a full understanding of the invention. However, the invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.
[0027] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.
[0028] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."
[0029] The present invention will now be described in further detail with reference to the accompanying drawings: To better illustrate the technical effects of the present invention, the present invention provides the following specific embodiments to illustrate the above technical process: Example 1: A method for implementing self-evolutionary learning of an intelligent agent, such as... Figure 1 As shown, it includes the following steps: I. Evolutionary Closed-Loop Design Step 1: Perceive the environment: The agent collects the environmental state (such as text, images, and numerical features) through sensors / data interfaces and outputs a standardized state vector.
[0030] Step 2, Execute the action: Based on the current policy π, select the optimal action a (such as task operation or decision instruction) from the action space A.
[0031] Step 3: Obtain feedback: Collect environmental response r (instant reward / penalty), task completion rate, etc., and generate multi-dimensional feedback signals.
[0032] Step 4: Optimize the model: Adjust the strategy / model parameters using feedback signals to minimize the loss or maximize the cumulative reward.
[0033] Step 5: Update the strategy: Solidify the optimized parameters into the new strategy π', and store the experience (S,a,r,S') in the experience base.
[0034] Step 6, Experience Reuse: In subsequent iterations, prioritize the use of high-value experiences to reduce repeated trial and error and accelerate evolution.
[0035] II. Learning-driven Step 1, Core Learning: Focusing on reinforcement learning, core strategies are optimized through trial and error and feedback.
[0036] Step 2, Unsupervised mining: Extract hidden patterns (such as task patterns and environmental features) from unlabeled interaction data and supplement with supervision signals.
[0037] Step 3, Capability Transfer: Transfer the parameters / features learned from the source task (such as text classification) to the target task (such as summary generation) to reduce the learning cost.
[0038] Step 4: Lifelong learning: Retain historical abilities while learning new tasks, avoid catastrophic forgetting, and continuously expand the boundaries of your abilities.
[0039] Proximity strategy optimization algorithm: Initialize PPONet (Actor-Critic dual-head network) and optimizer, and set parameters such as state dimension and action dimension.
[0040] The intelligent agent interacts with the environment, collects data such as environmental state (S), actions (a), and feedback signals (r), and stores them as experience samples.
[0041] Calculate the cumulative return after discounts: Gt = r t +γr t+1 +γ²r t+2 +…, standardized as the learning objective; γ is the discount factor; t represents time; Iterative update strategy: The probability distribution of new actions is output by the network, the probability ratio with the old strategy is calculated, the update range is limited by a clamping function, and the total loss is optimized by combining Actor loss (strategy optimization) and Critic loss (value assessment).
[0042] Repeat the iteration until the policy converges, and output the optimal policy as the pre-trained policy π.
[0043] Transfer learning optimization algorithm: Load the pre-trained backbone model (such as an LLM encoder), freeze all its parameters, and avoid losing historical capabilities.
[0044] Add a LoRA low-rank adaptation layer (containing low-rank matrices A and B and scaling factor α) after the backbone model, and train only the parameters of this layer.
[0045] The features / parameters learned from the source task are transferred to the target task, and the feature output is fine-tuned through the LoRA layer to adapt to the requirements of the new task.
[0046] By combining the target task data with the training of the classification head, the ability transfer from the source task to the target task is completed.
[0047] III. Memory and Cognitive Support Step 1, Experience Storage: Transform the interaction experience (S, a, r, S') and knowledge rules into vectors and store them in the memory system. S represents the environmental state after performing action a.
[0048] Step 2, Knowledge Structuring: Use knowledge graphs to organize the connections between concepts (such as task-action-effect mapping) to form structured knowledge.
[0049] Step 3, Memory Retrieval: Based on the current task requirements, retrieve relevant experience / knowledge from long-term memory to assist in decision-making.
[0050] Step 4, Dynamic Update: Delete invalid experiences, strengthen the weight of high-value memories, and maintain the timeliness of the memory system.
[0051] Specific steps of the Long Memory Module (LSTM) algorithm Initialize the model: Set the input dimension (e.g., 32), hidden layer dimension (default 64), and number of LSTM layers (default 2), and build the LSTM network (batch_first=True, process data in batches).
[0052] Prepare input data: Organize the time series interaction data (vector) into tensors in the format [batch_size, seq_len, input_dim] (e.g., 32 batches, 10 time steps, 32-dimensional input).
[0053] Initialize hidden state: If no hidden state (h0, c0) is passed in, an initial hidden state and cell state of all zeros will be automatically generated with dimensions [num_layers, batch_size, hidden_dim].
[0054] Extracting long-term memory features: Input data is fed into the LSTM network, and the feature vectors of all time steps and the final hidden state are output; the output of the last time step (out[:, -1, :]) is taken as the long-term memory representation (memory_feat).
[0055] Output: Returns long-term memory features and the final hidden state, which can be used for subsequent experience storage or decision assistance.
[0056] Specific steps of the historical experience retrieval algorithm (VectorDB) Initialize the vector database: Set the vector dimension (e.g., 64), create a FAISS index based on L2 distance, and initialize an empty list to store the original experience vectors.
[0057] Storing experience vectors: The agent's interaction experience (S,a,r,S') is transformed into a NumPy array in [n, dim] format (e.g., 1000 experience points, 64-dimensional vector), and stored in the database using the add method (synchronously updating the index and the original vector list).
[0058] Construct retrieval query: Transform the current task / environment state into a query vector in [1, dim] format (consistent with the dimension of the storage vector).
[0059] Retrieve similar experiences: Call the search method, pass in the query vector and the number of similar results to be returned (top_k, default 5), calculate the similarity using L2 distance, and return the top_k most similar experience vectors and their corresponding distances.
[0060] Output search results: Return a list of similar experiences, allowing the agent to reuse high-value historical experiences and reduce repeated trial and error.
[0061] IV. Feedback and Reward Mechanism Step 1, Feedback Collection: Collect feedback signals from multiple sources (real-time environmental feedback, final task benefits, and self-correction signals).
[0062] Step 2, Feedback Normalization: Standardize the feedback from different dimensions to eliminate differences in units.
[0063] Step 3, Reward Integration: Taking into account both immediate rewards (short-term gains) and long-term rewards (task objectives), a comprehensive reward signal R_total is generated.
[0064] Step 4, Signal Transformation: Transform R_total into a loss value that the model can optimize.
[0065] Reward model training algorithm: Initialization: Construct a network with two linear layers and ReLU activation. The input is the feature vector dimension, and the output is a single reward score.
[0066] Data preparation: "Preference data pairs" (32 sets of high-quality response vectors + corresponding low-quality response vectors) were used to simulate human judgment on the quality of the results.
[0067] Loss calculation: The model learns "high-quality response reward > low-quality response reward" through the loss function - calculate the logarithm of the sigmoid value of the difference between the two rewards, and take the negative average as the loss (the larger the reward difference, the smaller the loss).
[0068] Training iterations: 100 iterations, optimizing network parameters through backpropagation, allowing the model to gradually learn "what is a good action / response".
[0069] Feedback reward implementation algorithm: Core Process Policy network initialization: Construct a fully connected network, take the environment state dimension as input, and output the probability distribution of each action (softmax normalization).
[0070] Comprehensive reward calculation: Normalized multi-source feedback (user rating 1-5 stars → 0-1, response time 1-8 → 0.2-1); The task reward (success rate, weight 0.7) and feedback reward (explicit + implicit, weight 0.3) are combined according to their weights to generate R_total.
[0071] Strategy Training: Simulate environmental conditions and random action selection, and collect feedback data (success rate, score, response time); Calculate the policy loss (negative logarithmic action probability × overall reward) and update the policy network parameters through backpropagation.
[0072] Iterative optimization: Iterate 500 times to gradually shift the strategy towards actions that result in "high task success rate, high user rating, and short response time".
[0073] V. Stability and Generalization Guarantee Step 1, Prevent overfitting: Avoid model fitting to training data through regularization and data augmentation.
[0074] Step 2, Anti-forgetting: Protect core parameters of historical tasks and retain existing capabilities during new task learning.
[0075] Step 3, Generalization Optimization: Optimize the model initialization method to improve the speed of cross-scene and cross-task adaptation.
[0076] Step 4, Iterative Validation: After each round of evolution, the performance is validated using a test set to avoid model degradation.
[0077] Specific steps of the Elastic Weight Consolidation (EWC) algorithm to resist forgetting 1. Initialize EWC (Preparation for core parameter protection) Pass in key components: When initializing the EWC class, pass in the model to be protected, the historical task dataset loader, and the running device (default CPU).
[0078] Filter trainable parameters: Extract the parameters in the model that require gradient updates (requires_grad=True) and store them as a parameter dictionary (self.params).
[0079] Initialize the Fisher matrix: Create an all-zero Fisher information matrix (self.fisher) with the same dimensions as the trainable parameters to record the importance of key parameters in historical tasks.
[0080] Calculate Fisher information: Call the _compute_fisher method to calculate the Fisher value (expectation of the squared gradient) of each parameter based on historical task data, and mark the key parameters.
[0081] 2. Calculate the Fisher information matrix (mark key parameters) Switch the model to evaluation mode: Set model.eval() to avoid regularization and other operations in training mode affecting gradient calculation.
[0082] Traverse historical task data: Iterate through the historical task dataset, moving the input data (x) and labels (y) to the specified device.
[0083] Calculate and accumulate gradients: For each batch of data, calculate the model output and cross-entropy loss, backpropagate to obtain the parameter gradients, accumulate the squared gradients into the Fisher matrix, and finally divide by the dataset length to obtain the expected value of the squared gradient.
[0084] 3. Calculate the EWC penalty term (to protect historical parameters). Compare current and historical parameters: Iterate through the model's currently trainable parameters and calculate the difference between them and the historical task parameters (self.params) stored during initialization.
[0085] Weighted calculation of penalty term: The difference between parameters is weighted and summed using the Fisher matrix (parameter importance) to obtain the EWC penalty term. The difference between key parameters will result in a larger penalty.
[0086] 4. Combine with EWC's training process (new task training + anti-forgetting) Switch the model to training mode: Set model.train() to enable gradient calculation.
[0087] Traverse the new task data: Iterate over the new task dataset, move the data to the specified device, and initialize the optimizer gradient to zero.
[0088] Calculate the double loss term: Task loss: The cross-entropy loss of the new task, ensuring the learning effect of the new task; EWC loss (ewc_loss): The product of the penalty term and the weight coefficient (lambda_ewc, default 1e3), which restricts the variation of key parameters.
[0089] Total loss optimization: The total loss is the sum of the task loss and the EWC loss. The model parameters are updated through backpropagation. While learning new tasks, the key parameters of historical tasks are protected from being significantly modified, thus avoiding forgetting.
[0090] 5. Iterative Training and Convergence Repeat the training process for the new task, calculate and optimize the double loss term in each iteration until the model converges on the new task, thus achieving anti-forgetting training that combines "learning new capabilities + retaining old capabilities".
[0091] This invention adopts an evolutionary closed-loop design: the basic closed loop is based on the process of perceiving the environment → performing actions → obtaining feedback → optimizing the model → updating the strategy, and superimposed with a sub-closed loop of experience storage-reuse-update to achieve continuous iteration.
[0092] This invention adopts a learning-driven approach: reinforcement learning is the core (combined with policy gradient and Q-learning), coupled with unsupervised / weakly supervised learning to mine data patterns, and then cross-scenario evolution is achieved through transfer learning and lifelong learning to avoid repetitive learning.
[0093] This invention employs memory and cognition support: it uses a vector database and knowledge graph to store experience and knowledge, and uses an LSTM / Transformer architecture to enable long-term memory retrieval, dynamically update experience weights, and ensure the continuity of evolution.
[0094] This invention employs a feedback and reward mechanism: an adaptive reward function is designed (taking into account both immediate and long-term benefits), supporting the fusion of multiple sources such as environmental feedback, task result feedback, and self-correction feedback, which are then accurately converted into learning signals.
[0095] This invention employs stability and generalization guarantees: it prevents overfitting and ability forgetting through regularization, experience replay, and incremental training, and combines meta-learning to optimize model initialization and improve cross-domain adaptation speed.
[0096] The present invention provides a method for implementing self-evolutionary learning of an intelligent agent with the following advantages: 1. Autonomous feedback loop: It can automatically collect environmental / task result feedback without human intervention and convert it into learning signals.
[0097] 2. Dynamic experience reuse: Effective experience is stored through a memory mechanism, and the experience base can be called up and updated as needed to avoid repeated trial and error.
[0098] 3. Unsupervised / weakly supervised learning capability: Free from dependence on large-scale labeled data, autonomously discovering patterns from unlabeled data and interactive data.
[0099] 4. Anti-forgetting and stability: It does not lose its existing capabilities during the evolution process and maintains model stability through techniques such as regularization and experience replay.
[0100] 5. Evolution of generalization capabilities: Not only can it optimize the performance of the current task, but it can also transfer capabilities to new scenarios and tasks, achieving cross-domain adaptation.
[0101] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules, units, or units is merely a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units, modules, or components may be combined or integrated into another device, or some features may be ignored or not executed.
[0102] The units may or may not be physically separate. The components shown as units can be one or more physical units, meaning they can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
[0103] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.
[0104] In particular, according to embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by a central processing unit (CPU), it performs the functions defined in the methods of this invention. It should be noted that the computer-readable medium described above in this invention can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof.
[0105] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0106] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for implementing self-evolutionary learning of an intelligent agent, characterized in that, Includes the following steps: Obtain the environment status; The environmental state is input into the pre-trained policy, which outputs the optimal action and executes it. After the action is performed, a feedback signal is obtained; the feedback signal includes environmental response and task completion status. The strategy is optimized based on the feedback signals to obtain a new strategy.
2. The method for implementing self-evolutionary learning of an intelligent agent according to claim 1, characterized in that: The training method for the strategy employs a near-end strategy optimization algorithm; The near-end strategy optimization algorithm specifically includes the following steps: Obtain historical data; Based on historical data, calculate the cumulative return of the discount and standardize it as the learning objective; based on the learning objective, obtain the Actor loss and Critic loss; The probability distribution of new actions is output through the network, and the probability ratio with the old strategy is calculated. The total loss is then optimized by combining the Actor loss and Critic loss. Repeat the iteration until the policy converges, and output the optimal policy as the pre-trained policy.
3. The method for implementing self-evolutionary learning of an intelligent agent according to claim 2, characterized in that: The strategy is optimized using a transfer learning optimization algorithm. The transfer learning optimization algorithm includes: Load the pre-trained backbone model and freeze all its parameters; Add a LoRA low-rank adaptation layer after the backbone model, and train only the parameters of this layer; The features / parameters learned from the source task are transferred to the target task, and the feature output is fine-tuned through the LoRA low-rank adaptation layer to adapt to the requirements of the new task. By training a classification head using data from the target task, the ability transfer from the source task to the target task is achieved.
4. The method for implementing self-evolutionary learning of an intelligent agent according to claim 3, characterized in that, It also includes the following steps: Based on the environmental state, actions, feedback signals, environmental state after executing action a, and knowledge rules, vectors are transformed to generate long-term memory features. By structuring the characteristics of long-term memory, structured knowledge can be obtained. Based on the current task requirements, long-term memory features are extracted from structured knowledge as historical data.
5. The method for implementing self-evolutionary learning of an intelligent agent according to claim 4, characterized in that, The long-term memory features employ a long-term memory algorithm: Define the input dimension, hidden layer dimension, and number of LSTM layers, and construct an LSTM network; The vectors are fed into the LSTM network, which outputs the feature vectors at all time steps as long-term memory representations.
6. The method for implementing self-evolutionary learning of an intelligent agent according to claim 5, characterized in that, The strategy is optimized based on the feedback signals to obtain a new strategy, which specifically includes the following steps: Obtain feedback signals; The feedback signal is normalized to obtain the normalized result; The normalized results are weighted and fused according to preset weights to obtain a comprehensive reward signal; Use the comprehensive reward signal as an optimizable loss value; The strategy is optimized based on the optimizable loss value.
7. The method for implementing self-evolutionary learning of an intelligent agent according to claim 6, characterized in that, After obtaining the environment status, the following steps are also included: The environment state is preprocessed through regularization and data augmentation.
8. The method for implementing self-evolutionary learning of an intelligent agent according to claim 7, characterized in that, It also includes the following steps: Protect historical task parameters during new task learning; Performance is validated using a test set after each round of evolution.
9. A system for implementing self-evolutionary learning of an intelligent agent, used to implement the self-evolutionary learning of an intelligent agent as described in any one of claims 1-8, characterized in that, include: The acquisition module is used to acquire environmental status. The execution module is used to input the environmental state into the pre-trained policy, output the optimal action, and execute it. The feedback module is used to acquire feedback signals after an action is performed; the feedback signals include environmental response and task completion status. The optimization module is used to optimize the strategy based on feedback signals to obtain a new strategy.