An agent meta-learning method, device, apparatus, and medium

Abstract

The application discloses an agent meta-learning method, device, equipment and medium, the method comprises the following steps: driving the agent to execute the task based on the received task request and the hierarchical memory library, and collecting the feedback signal in the execution process; according to the feedback signal, the execution result is checked for credibility, and a verified structured execution track is generated; the track is evaluated for value to filter out high-value track execution experience compression, the execution experience of the task level is converted into a reusable skill template of the task family level, and stored in the hierarchical memory library; according to the reusable skill template in the hierarchical memory library, the strategy optimization task is updated differently through a double-time-scale scheduling mechanism, and an optimization result is generated; the verified strategy version is deployed to the online environment through the multi-stage safety management pipeline for verification and release. The cross-task experience reuse rate is improved, the storage occupation and response delay are reduced, and safe, stable and low-cost strategy self-evolution is realized.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a meta-learning method, apparatus, device, and medium for intelligent agents. Background Technology

[0002] With the rapid development of artificial intelligence technology, intelligent agent systems based on large language models have been widely used in fields such as financial customer service, software engineering and office automation. Intelligent agent systems need to automatically generate execution plans based on user requests and call external tools to complete tasks.

[0003] Existing technologies include: Static configuration-based agent methods: Before system deployment, technicians determine configuration parameters such as prompt templates, tool routing rules, and workflow steps through offline debugging; after deployment, the system is loaded and run in read-only mode. Self-reflection-based agent methods: After each task execution, the agent model performs a text-based debriefing analysis of its own output, generating "reflection text" which is then appended to the context of subsequent tasks. Reinforcement learning-based policy optimization methods: The agent's multi-step tool invocation process is modeled as a Markov decision process; by defining a reward function, end-to-end weight updates are performed on the policy model using proximal policy optimization or similar algorithms.

[0004] However, with static configuration, the system cannot automatically adjust configuration parameters based on the execution trajectory when task distribution shifts or new failure modes emerge. In self-reflection mechanisms, the evaluation results of the model's own output are affected by the model's parameter biases, easily leading to "self-consistency bias." This results in unstable guidance of the reflection text for subsequent tasks and may even introduce new noise. Furthermore, the reward function design in reinforcement learning relies heavily on manual annotation and domain knowledge. In real-world business scenarios, reward signals are sparse and delayed, causing slow training convergence. Summary of the Invention

[0005] This invention provides a meta-learning method, apparatus, device, and medium for intelligent agents to improve the reusability of cross-task experience and achieve safe, stable, and low-cost policy self-evolution.

[0006] According to a first aspect of the present invention, a meta-learning method for an intelligent agent is provided, comprising:

[0007] Based on the received task request and the hierarchical memory bank, the agent executes the task and collects feedback signals from multiple heterogeneous information sources during the execution process.

[0008] The credibility of the execution result is verified based on the feedback signal, and a verified structured execution trajectory is generated.

[0009] The validated structured execution trajectories are evaluated for value to select high-value trajectories, and the execution experience of the high-value trajectories is compressed to transform the task-level execution experience into reusable skill templates at the task family level, which are then stored in the hierarchical memory bank.

[0010] Based on the reusable skill templates accumulated in the hierarchical memory bank, the strategy optimization task is assigned to the fast adaptation layer or the slow sedimentation layer for differentiated updates through a dual time-scale scheduling mechanism to generate optimization results.

[0011] Based on the optimization results, the policy is verified and released through a multi-stage security governance pipeline, and the verified policy version is deployed to the online environment.

[0012] According to a second aspect of the present invention, a meta-learning device for an intelligent agent is provided, comprising:

[0013] The signal determination module is used to drive the agent to execute tasks based on the received task request and hierarchical memory, and to collect feedback signals from multiple heterogeneous information sources during the execution process.

[0014] The trajectory generation module is used to verify the credibility of the execution result based on the feedback signal and generate a verified structured execution trajectory.

[0015] The template storage module is used to evaluate the value of the verified structured execution trajectory to select high-value trajectories, compress the execution experience of the high-value trajectories, and transform the task-level execution experience into reusable skill templates at the task family level, which are then stored in the hierarchical memory bank.

[0016] The strategy optimization module is used to allocate the strategy optimization task to the fast adaptation layer or the slow accumulation layer for differentiated updates based on the reusable skill templates accumulated in the hierarchical memory bank, and generate optimization results through a dual time scale scheduling mechanism.

[0017] The result determination module is used to verify and release the optimization results through a multi-stage security governance pipeline, and deploy the verified policy version to the online environment.

[0018] According to a third aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0019] At least one processor; and

[0020] A memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the meta-learning method of the intelligent agent according to any embodiment of the present invention.

[0022] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the meta-learning method of the intelligent agent according to any embodiment of the present invention.

[0023] According to a fifth aspect of the present invention, embodiments of the present invention also provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the meta-learning method of an intelligent agent according to any embodiment of the present invention.

[0024] The technical solution of this invention significantly improves the reliability and quality of execution data through a layered memory and multi-source feedback verification, providing a reliable foundation for subsequent learning. Through value assessment and experience compression, fragmented task experience is transformed into reusable task family skill templates, greatly increasing the cross-task experience reuse rate and reducing storage overhead. A dual-timescale scheduling mechanism enables a rapid adaptation layer to respond to business changes at the minute level and a slow accumulation layer to deeply optimize strategies, avoiding system instability caused by online updates. A multi-stage security governance pipeline ensures that strategy updates are fully verified and released incrementally, compressing fault recovery time to the minute level and significantly improving the stability and traceability of system deployment.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart of a meta-learning method for an intelligent agent provided according to Embodiment 1 of the present invention;

[0028] Figure 2 This is a schematic diagram of the architecture of a meta-learning method for an intelligent agent according to Embodiment 1 of the present invention;

[0029] Figure 3This is a schematic diagram of the structure of a meta-learning device for an intelligent agent according to Embodiment 2 of the present invention;

[0030] Figure 4 This is a schematic diagram of the structure of an electronic device that implements an embodiment of the present invention. Detailed Implementation

[0031] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0032] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0033] Example 1

[0034] Figure 1 This is a flowchart illustrating a meta-learning method for an intelligent agent according to Embodiment 1 of the present invention. This embodiment is applicable to the learning process of an intelligent agent. The method can be executed by a meta-learning device for the intelligent agent, which can be implemented in hardware and / or software and can be configured in an electronic device. Figure 1 As shown, the method includes:

[0035] S110. Based on the received task request and hierarchical memory, the agent is driven to execute the task, and feedback signals from multiple heterogeneous information sources are collected during the execution process.

[0036] In this embodiment, a task request refers to a business processing application message initiated by the end user / upstream system to the intelligent agent. For example, a task request message (fields: globally unique task ID task_id: UUID type, task family identifier task_family: enumeration type, task content input_payload: JSON format, constraint conditions constraint_list: string array, request initiation timestamp: timestamp type) can be submitted by the client to the API gateway's request interface via the HTTP / gRPC protocol. The hierarchical memory is a multi-level, structured data and knowledge storage system, divided into four levels: fragment memory, case memory, pattern memory, and skill memory. It is also equipped with an independent failure mode library, which can store short-term session context, long-term cross-session trajectory, historical task cases, general skill templates, and error anti-pattern data, etc. It is supported by a vector database (which may include skill / case / pattern vector retrieval), a relational database (skill library / failure mode library metadata), a distributed time-series storage (execution trajectory / real-time indicator stream), and a unified data access interface Redis (read / write / permission constraint) cache. Heterogeneous information sources refer to feedback sources that are independent of each other and have different data formats and acquisition dimensions. Feedback signals refer to structured evaluation data extracted from various heterogeneous information sources.

[0037] Specifically, the client submits a task request message via HTTP / gRPC. The processor first parses the task, classifies it into task families, and generates task family metadata (task difficulty, risk level, required tools, etc.). If the task is unlabeled, the processor automatically infers the task family based on preset rules. The processor performs a hybrid retrieval (vector similarity + structured tag filtering) in the hierarchical memory, matching the historical skill templates corresponding to the current task family. Based on the task difficulty and computing power budget, it automatically selects the appropriate large language model: a lightweight model for simple tasks and a high-capacity model for complex inference tasks. A standardized execution plan is generated by combining the retrieved skill templates, driving the agent to call external tool interfaces to complete business operations according to the execution plan's scheduling steps. After the task is executed, the feedback system synchronously collects raw data from multiple heterogeneous information sources and integrates it into a feedback signal.

[0038] S120. Verify the credibility of the execution results based on the feedback signal and generate a verified structured execution trajectory.

[0039] In this embodiment, the execution result refers to the result produced by the agent after performing the task. The structured execution trajectory refers to the data set after the complete task execution process of the agent is standardized and serialized, and may include fields such as task identifier, execution steps, tool call records, intermediate results, error logs, reflection and correction records, timestamps, and feedback data.

[0040] Specifically, the processor can perform credibility-weighted fusion of feedback signals to generate a preliminary structured feedback record. Based on this preliminary structured feedback record, multi-level credibility verification is performed on the execution results. The processor can uniformly encapsulate all data, including verified task information, execution steps, and timestamps, to generate a verified structured execution trajectory. Samples that fail verification are marked as unlearnable samples and directly intercepted.

[0041] S130. The validated structured execution trajectories are evaluated for value to select high-value trajectories, and the execution experience of high-value trajectories is compressed to transform task-level execution experience into reusable skill templates at the task family level and stored in a hierarchical memory bank.

[0042] In this embodiment, a high-value trajectory refers to a structured execution trajectory whose overall score and learning value score, after value assessment, exceed a preset threshold, thus providing a valid reference for agent optimization. A task family-level reusable skill template refers to a standardized and generalized execution template abstracted from the same type of business scenario (task family), which may include metadata such as triggering conditions, standard execution steps, verification nodes, and confidence levels.

[0043] Specifically, the processor can construct a multi-dimensional evaluation matrix based on fused feedback, calculate the score of each structured trajectory, and set a threshold. Trajectories exceeding the threshold are classified as high-value trajectories, while those below the threshold are discarded. The processor can perform attribution analysis, pattern abstraction, and templating on high-value trajectories, transforming task-level execution experience into reusable skill templates at the task family level. It then performs conflict detection between these skill templates and existing templates in a hierarchical memory, storing conflict-free skill templates in the hierarchical memory.

[0044] S140. Based on the reusable skill templates accumulated in the hierarchical memory bank, the strategy optimization task is assigned to the fast adaptation layer or the slow sedimentation layer for differentiated updates through a dual time scale scheduling mechanism, generating optimization results.

[0045] In this embodiment, the strategy optimization task refers to the computational task of iteratively updating the agent based on skill templates, historical trajectories, and failure modes within a hierarchical memory. The dual-timescale scheduling mechanism refers to the intelligent scheduling logic that divides the strategy optimization task into two channels: a fast adaptation layer and a slow sedimentation layer. The fast adaptation layer is a lightweight, high-real-time online optimization channel. The slow sedimentation layer is a heavyweight, high-complexity offline optimization channel. The optimization result refers to the two types of output generated after strategy optimization: the fast adaptation layer outputs online patches, and the slow sedimentation layer outputs complete candidate strategy versions.

[0046] Specifically, the processor can use a dual-time-scale scheduler to judge system load, task arrival rate, and knowledge base update information, dynamically determine the classification of the current policy optimization task, and assign the policy optimization task to the fast adaptation layer or the slow sedimentation layer for differentiated updates. The fast adaptation layer outputs fast adaptation patches, and the slow sedimentation layer outputs complete candidate policy versions, thereby obtaining the optimization results.

[0047] S150. Based on the optimization results, the policy version is verified and released through a multi-stage security governance pipeline, and the verified policy version is deployed to the online environment.

[0048] In this embodiment, the multi-stage security governance pipeline is a full-link security release process, consisting of: candidate generation, offline evaluation, sandbox simulation, canary release, A / B testing, official release, and monitoring rollback. A policy version is an independent version that has been snapshotted and solidified for the entire set of execution logic, configuration parameters, and skill library combinations of the intelligent agent. The online environment refers to the production and operation environment of the intelligent agent that provides formal services to end users.

[0049] Specifically, the processor can perform multi-stage security governance pipeline verification on the optimization results, including steps such as permission isolation, sandbox simulation, canary release and full audit. It can evaluate the current policy version offline, and then perform online evaluation after passing the evaluation. The policy version that passes online verification is deployed to the online environment, and anomaly monitoring and rollback are performed after the official release.

[0050] The technical solution of this invention significantly improves the reliability and quality of execution data through a layered memory and multi-source feedback verification, providing a reliable foundation for subsequent learning. Through value assessment and experience compression, fragmented task experience is transformed into reusable task family skill templates, greatly increasing the cross-task experience reuse rate and reducing storage overhead. A dual-timescale scheduling mechanism enables a rapid adaptation layer to respond to business changes at the minute level and a slow accumulation layer to deeply optimize strategies, avoiding system instability caused by online updates. A multi-stage security governance pipeline ensures that strategy updates are fully verified and released incrementally, compressing fault recovery time to the minute level and significantly improving the stability and traceability of system deployment.

[0051] Furthermore, based on the above embodiments, the steps of driving the agent to execute tasks based on the received task request and hierarchical memory, and collecting feedback signals from multiple heterogeneous information sources during execution, can be refined as follows:

[0052] Based on the task family identifier in the task request, retrieve historical planning templates matching the task family identifier from the hierarchical memory and determine the retrieval results; determine the target planning process based on the confidence level of the retrieval results and the historical planning templates; generate an initial execution plan based on the target planning process, drive the agent to execute tasks in the order of the steps in the initial execution plan, and select the corresponding computing power model instance according to the task complexity during execution; during execution, when a step execution failure is detected or the execution result deviates from the expected result, dynamically correct the subsequent execution steps according to the replanning mechanism, determine the corrected execution path and record the correction notes; continue to execute the task based on the corrected steps until the task is completed, and extract the process feedback signals in the execution trajectory, the environmental feedback signals in the environmental return messages, the rule feedback signals output by the rule engine, and the result feedback signals and preference feedback signals of the user interaction interface.

[0053] In this embodiment, the task family identifier is a metadata field used to classify tasks. Confidence is a quantitative indicator measuring the reliability of retrieval results, typically ranging from 0 to 1. The historical planning template is a structured sequence of steps extracted from past successful execution trajectories. The target planning process is the final plan used to guide the execution of the current task. The initial execution plan includes structured information such as plan identifier, step array, tool call chain, and rollback rules. Task complexity is comprehensively evaluated by indicators such as difficulty level, required inference depth, and expected token consumption. Computational model instances refer to large language models of different scales and capabilities (e.g., lightweight models for simple classification, and strong inference models for complex code generation). The replanning mechanism is a runtime exception handling mechanism; unlike replanning from scratch, it only makes local adjustments to subsequent steps of the current plan. Process feedback signals include execution efficiency indicators such as step redundancy, tool call success rate, number of retries, and execution latency. Environmental feedback signals originate from messages returned by the tool manager after calling external interfaces, including return codes, return load, and execution time. Rule feedback signals originate from compliance judgment frames pushed by a pre-defined external rule service (such as whether data anonymization is complete, or whether the operation is within the allowed whitelist). Result feedback signals indicate whether the final output of the task meets user requirements or acceptance criteria. Preference feedback signals refer to the subjective evaluation of the output results by users or evaluators.

[0054] Specifically, the processor can receive task request messages (input), perform verification and mapping (processing) of task type identifiers (such as the `task_family` field in the example above), query the task distribution registry, and output a task family metadata frame (e.g., fields: `task_family_id`: string, `difficulty_level`: integer, `risk_level`: enumeration type, `required_tools`: string array). If `task_family` is not labeled, inference is performed based on preset classification rules. Task classification and mapping are existing technologies and will not be elaborated upon. It receives the task family metadata frame and constraint list (input), selects the most suitable LLM instance according to the `difficulty_level` and `cost_limit` fields based on a preset strategy (e.g., lightweight model for simple tasks, strong inference model for complex tasks), calculates the computing power budget based on the difficulty rating, determines the computing power matching model instance for the task according to preset rules, and outputs a model selection instruction. The system receives task family metadata frames (input), performs a retrieval based on key information from the current session's input and model feedback during short-term memory dialogues in the data and knowledge layer, and from all previous historical dialogues in long-term memory. It employs a hybrid retrieval method combining skill memory, pattern memory, and case memory (e.g., skill memory followed by pattern memory followed by case memory). The retrieval method combines vector similarity matching (for semantic understanding) with structured field filtering (for precise matching of task family identifiers), outputting a retrieval result set (fields: candidate skills_candidates: object array, match_score: floating-point number, source_trace_ids: UUID array). Based on the retrieved skill templates, it generates an execution plan data frame (e.g., fields: plan_id: UUID, steps: object array, tool_chain: string array, fallback_rules: object array). If the retrieval result set is empty or the match_score is below a preset threshold, a default planning process is triggered, and the execution plan data frame is written to an execution plan queue (e.g., a Redis message queue), determining the target planning process.

[0055] In real-world computer system operation, if an agent generates an execution plan using completely independent end-to-end inference for each task, it leads to a large amount of redundant computation. Furthermore, when tasks belong to the same category, their planning steps often exhibit high similarity. Therefore, by introducing a task family mapping mechanism, input requests are categorized into a predefined task distribution space. This allows the system to prioritize retrieving and reusing historically validated planning templates for that type of task, significantly reducing online inference overhead and redundant planning computation. Simultaneously, since different tasks have varying computational resource requirements (e.g., simple queries and complex code generation tasks require different model capabilities), matching the optimal computing power model based on task complexity avoids excessive calls to high-computing-power models while meeting task requirements, preventing computational waste and thus reducing system token consumption and response latency.

[0056] Specifically, an initial execution plan is generated based on the goal planning process. The agent can schedule the execution of steps sequentially in the `steps` array of the initial execution plan (the aforementioned plan data frame), outputting a step execution state stream (fields: `step_id`: string, `status`: enumeration type, `intermediate_result`: JSON format, `latency_ms`: integer). Step scheduling and state management are existing technologies and will not be elaborated upon. Based on the tool call request (input) in the step execution state stream, external tool interfaces (such as database adapters, code executors, or search engines) are called via HTTP / gRPC / MQTT protocols. The external tool call protocol is based on existing technology and will not be elaborated upon. It utilizes the tool's return message (fields: tool_name: string, return_code: integer, return_payload: JSON format, execution_time: integer) and the execution state flow and tool return message (input) to dynamically adjust steps in long-chain tasks, rather than rigidly executing the initial plan. It also includes a real-time replanning mechanism to attribute failures and correct the plan when key steps or results deviate from the expected outcome (task failure). Based on the corrected final execution result, correction records, error types, and timestamps (input), structured serialization is performed to generate the original execution trajectory conforming to the execution trajectory data schema (fields: trace_id: UUID, risk ID task_id: UUID, plan_id: UUID, step_records: array of objects, tool_invocations: array of objects, error_log: array of objects, final_output: JSON format, start timestamp_start: timestamp, end timestamp_end: timestamp, reflection_notes: object). The final_output is then checked for format compliance and security filtering (toxicity detection / PII desensitization). If it fails to meet the standards, the execution steps are reverted to the order of steps in the steps array and written to a distributed time-series storage (such as a columnar storage engine). Process feedback signals (step redundancy, tool call success rate) can be extracted from the original execution trajectory; environment feedback signals (API return code, test results) can be extracted from the environment return message; rule feedback signals (compliance boolean value, violation type) can be extracted from the rule engine output; and result feedback signals and preference feedback signals can be extracted from the user interaction interface.

[0057] When an agent executes a multi-step task, if it strictly follows the initial plan without allowing mid-process corrections, the entire task chain will be forced to terminate if a step fails due to environmental changes or tool malfunctions, resulting in wasted computing resources. Therefore, a replanning mechanism is introduced during execution, similar to exception handling in an operating system. It can detect execution deviations at runtime and trigger local replanning, thereby improving task success rate. Furthermore, to support subsequent meta-learning, the system must persistently store the entire execution process in a structured format, rather than just retaining the final result. By defining a unified schema, heterogeneous data such as step records, tool calls, error logs, and correction notes are integrated into a standardized trajectory, enabling subsequent learning layers to efficiently retrieve and analyze historical behavior like querying a database. The GuardrailGate setting prevents output that does not meet security specifications or format requirements from being directly returned to the caller, acting as a filtering mechanism similar to a network firewall. By introducing the replanning mechanism, dynamic error correction and compliance interception during execution are achieved, significantly reducing the token consumption ratio of invalid executions and minimizing the risk of erroneous output contamination.

[0058] Furthermore, based on the above embodiments, the steps of verifying the reliability of the execution results according to the feedback signals and generating a verified structured execution trajectory can be refined as follows:

[0059] The feedback signals are weighted by credibility to generate structured feedback records. The execution results and structured feedback records are then verified sequentially through a layered verification pipeline, which includes: preset logic rule verification, unit test verification, external fact verification, simulator environment interaction verification, multi-review voting verification, and manual review verification. At each verification level, if the execution result fails the current level's verification, the failed execution trajectory is marked as unlearnable and prevented from entering the subsequent experience compression stage; otherwise, the execution trajectory that passes all verification levels is marked as a reliable trajectory, and a structured execution trajectory is determined.

[0060] In this embodiment, the structured feedback record is a record organized according to a unified data pattern, consisting of weighted multi-source feedback. It includes trajectory identifiers, original data from each feedback source, confidence scores, and timestamps. An unlearnable state is a marker indicating that the trajectory is unsuitable for meta-learning or policy optimization. A reliable trajectory refers to an execution trajectory that has passed all six levels of the hierarchical verification pipeline; it is considered to have high data quality and reliable feedback signals, making it suitable as input for meta-learning.

[0061] Specifically, execution can be scheduled according to the planned data frame (input), following the order of steps in the `steps` array, and the execution status stream of each step can be output (fields: `step_id`: string, `status`: enumeration type, `intermediate_result`: JSON format, `latency_ms`: integer). Step scheduling and status management are existing technologies and will not be elaborated upon. The processor can preset a basic confidence level for each type of feedback signal, perform confidence weighting on the five types of feedback signals, and generate structured feedback records (fields: `trace_id`, `feedback_sources` array, `env_feedback`, `rule_feedback`, `result_feedback`, `process_feedback`, `preference_feedback`, `confidence_scores` array). The system performs multi-level verification (processing) on ​​the original execution trajectory and structured feedback records, including preset logic rule verification, unit test verification, external fact-checking, simulator environment interaction verification, multi-review voting verification, and manual review verification. It then outputs a verification decision frame, marking trajectories that fail verification as unlearnable and preventing them from entering the subsequent experience compression stage. It receives resource consumption data (input) from the step execution state stream, performs token counting and API call cost accumulation (processing), and outputs resource consumption records (fields: trace_id: UUID, token_count: integer, api_cost: floating-point, latency_budget_status: enumeration type). Resource metering is existing technology and will not be elaborated upon. Finally, the structured feedback records (fields: trace_id, feedback_sources, env_feedback, rule_feedback, result_feedback, process_feedback, preference_feedback, confidence_scores) and the verification decision frames (fields: trace_id, verification_status, failed_checks array, adjusted_confidence) are written to the feedback database (such as PostgreSQL relational storage).

[0062] In the training process of machine learning systems, relying solely on feedback signals from a single source (such as user ratings) can easily lead to incorrect model updates due to sparsity or subjective bias. This method, however, collects multi-dimensional signals, including environmental feedback (actual tool execution results), rule feedback (compliance judgment), and process feedback (execution efficiency indicators), to form a comprehensive evaluation of task completion quality. This is similar to using multiple sensors simultaneously for state observation in a control system, thereby improving observation accuracy. The hierarchical validation pipeline solves the self-judgment problem.

[0063] Furthermore, based on the above embodiments, the steps of evaluating the value of verified structured execution trajectories to select high-value trajectories, compressing the execution experience of high-value trajectories, and transforming task-level execution experience into reusable skill templates at the task family level and storing them in a hierarchical memory bank can be further refined as follows:

[0064] Based on the validated structured execution trajectories, the trajectories are valued to select high-value trajectories; attribution analysis is performed on the high-value trajectories to locate key decision points and obtain attribution results; task processing patterns are abstracted from the attribution results to generate reusable skill templates; and the skill templates are registered in a hierarchical memory bank based on a conflict resolution mechanism.

[0065] In this embodiment, the attribution result can be understood as the structured analysis data output after systematically tracing the reasons for success or failure in high-value execution trajectories. The task processing pattern refers to the processing rules with cross-task applicability retained after stripping personalized information from the attribution result. The conflict resolution mechanism refers to the rule-based process by which the system automatically detects and handles conflicts when a newly generated skill template overlaps with an existing skill template in the hierarchical memory bank in terms of triggering conditions but contradicts in execution steps.

[0066] Specifically, based on the evaluation matrix and error classification system of five types of feedback (insufficient knowledge / reasoning error / retrieval error / execution error), a multi-dimensional evaluation matrix is ​​calculated on the verified structured execution trajectory and the above-mentioned multiple feedback signals. According to the confidence score (confidence_scores) and dimension weights in the structured feedback record, the overall score (overall_score) is calculated, and the error category (error_category) is determined according to the error log (error_log) and feedback sources (feedback_sources), thus outputting the evaluation result frame (fields: trace_id: UUID, overall score (overall_score): floating point, dimension scores (dimension_scores): JSON format, error_category: enumeration type, learning value (learning_value): floating point). Based on the evaluation result frame, high-value experience is filtered and buffered for the execution trajectory. Trajectories with an overall_score higher than a preset threshold or a learning_value higher than a preset threshold are marked as high-value experiences and written to the buffer queue. Trajectories that fail the Verifier verification or have an overall_score lower than the elimination threshold are discarded, and high-value experience entries are output (fields: buffer_id: UUID, trace_id: UUID, overall_score: floating point, learning_value: floating point, compressed_trace: JSON format, expiration_time: timestamp, priority_tag: string).

[0067] In continuously running systems, a large number of trajectories may be repetitive routine operations or low-quality erroneous attempts. Feeding all this data into the learning module would not only waste computational resources but could also cause policy updates to deviate due to excessive noise. Therefore, this step essentially sets up a quality control gate before the data enters the core learning algorithm: trajectories are quantitatively evaluated using a multi-dimensional scoring system to distinguish between samples worth learning and those that should be discarded; an expiration mechanism is further introduced because old experiences may become invalid over time due to changes in business rules or environmental migration, similar to the TTL (Time to Live) mechanism in a caching system, preventing outdated data from polluting the current policy optimization process. Through an evaluation matrix and value screening mechanism, high-value learning samples are automatically identified from massive execution trajectories, reducing the interference of noisy data on the meta-learning process, lowering the peak memory usage of experience buffer storage, and optimizing the processing order through priority labels.

[0068] Specifically, based on high-value experience entries, the compressed trajectories within these entries are analyzed to identify the reasons for success / failure, pinpoint key decision points, abstract task processing patterns from the attribution results, generate pattern memory entries, and transform these entries into reusable skill templates. These templates include trigger conditions, execution steps, and verification points, outputting candidate skill entries (fields: skill_id: UUID, trigger_condition: JSON format, execution_template: JSON format, verification_points: string array, confidence: floating-point number, source_trace_ids: UUID array, creation_time: timestamp). Conflicts between candidate skill entries and the existing skill library (overlapping trigger conditions and contradictory execution steps) are detected. Entries without conflicts are directly registered, while conflicting entries are marked as pending review. Output knowledge base update frames (fields: update_type (enumeration types: insert / update / drop), target_library (enumeration types: skill / case / failure), merged_data (JSON format), conflict_resolution_log (JSON format).

[0069] The original execution trajectory typically contains a large amount of contextual information unrelated to the task's essence (such as specific timestamps and temporary variable values). Directly using this raw data to guide subsequent tasks not only incurs enormous storage overhead but also results in extremely poor generalization ability. The core idea of ​​this step is data compression and knowledge enhancement: by identifying the key decision points that determine the success or failure of a task through attribution analysis, and then by abstracting patterns to remove the personalized information of specific cases while retaining the processing rules that are universal across tasks, it is ultimately solidified into a skill template. This process is similar to compiling program source code into a reusable function library—callers do not need to care about the specific implementation details of the function, but only need to know the input conditions and output interfaces. At the same time, the system specifically maintains a failure mode library, the technical purpose of which is to establish immune memory: when the system encounters similar error triggers again, it can identify and avoid them in advance, rather than repeatedly trying and failing. The conflict resolution mechanism is to avoid the system falling into logical confusion when old and new experiences contradict each other (for example, old rules require A to precede B, while new experience indicates that B precedes A). By compressing the original execution trajectory into structured skill templates and failure modes, we can abstract from task-specific experience to task-family patterns, improve the reusability of cross-task experience, reduce repeated trial and error, and enhance the robustness of the system by using a failure mode library.

[0070] Furthermore, based on the above embodiments, the steps of allocating strategy optimization tasks to the fast adaptation layer or the slow accumulation layer for differentiated updates and generating optimization results, based on reusable skill templates accumulated in the hierarchical memory and through a dual-time-scale scheduling mechanism, can be refined as follows:

[0071] Based on a dual-time-scale scheduling mechanism, combined with the type and urgency of the policy update target and the current system load, a differentiated update layer is determined to be triggered. The urgency is determined based on abnormal signals from external business indicators. When a rapid adaptation layer is triggered, an online lightweight optimizer is invoked to generate a configuration-level patch, which is then used as the optimization result. When a slow sedimentation layer is triggered, an offline heavyweight optimizer is invoked to aggregate multiple skill templates accumulated in the hierarchical memory to generate candidate policy versions, which are then used as the optimization result.

[0072] In this embodiment, the type of policy update target refers to the seven types of policy parameters that need to be optimized, including prompt templates, few-shot examples, tool routing rules, workflow graphs, memory / skill libraries, evaluator parameters, and model weights. Urgency level: The update priority determined based on external business indicator anomaly signals, used to determine how quickly the policy update needs to take effect. Urgency levels are divided into high, medium, and low. External business indicator anomaly signals refer to externally monitored KPI anomaly alarm signals, such as decreased conversion rates, increased SLA default rates, and a sharp drop in user satisfaction scores. Current system load refers to the resource usage status of the system during runtime, including indicators such as task arrival rate, CPU / GPU utilization, memory usage, and API call queue length. Online lightweight optimizers refer to lightweight optimization algorithms that do not require large amounts of training data and computational resources, including the Prompt optimizer (DSPy / APE), the tool routing optimizer (Bandit algorithm), and the workflow evolver (graph editor). Configuration-level patches are lightweight update packages that modify system configuration parameters without changing the weights of the core model. They include fields such as patch identifier, target module (hint template / routing / workflow), patch content, and rollback snapshot identifier. Offline heavyweight optimizers are heavyweight optimization algorithms that require a large amount of training data and computational resources, including Distiller, Preference Optimizer (DPO / RLHF), Reinforcement Learning Trainer (PPO), and Population Evolver. Candidate policy versions are heavyweight update packages output by the slow sedimentation layer, containing fields such as policy version identifier, policy package (JSON format), benchmark report, and training metadata. They must undergo multi-stage security governance pipeline verification in Phase 3 before being deployed.

[0073] Specifically, the dual-timescale scheduler can determine whether to trigger the fast adaptation layer (online) or the slow sedimentation layer (offline) based on the urgency of the knowledge base update frame and system load indicators, and output scheduling instructions (fields: update_path: enumeration type, priority: integer, deadline: timestamp). The fast adaptation layer receives the scheduling instructions and the knowledge base update frame (input), performs online lightweight optimization (processing), and outputs a fast adaptation patch (fields: patch_id: universally unique identifier UUID, target_module: enumeration type [Prompt / Router / Flow], patch_content: JSON format, rollback_snapshot_id: universally unique identifier). Optimization methods include the prompt template optimizer (DSPy framework / automatic prompt project APE), the tool router opt (Bandit algorithm), and the workflow evolver (graph editor); these algorithms are existing technologies and will not be elaborated, but in this invention they are encapsulated as a constrained online patching mechanism. The slow sedimentation layer receives scheduling instructions and accumulated knowledge base update frames (input), performs offline heavy optimization (processing), and outputs candidate policy versions (fields: policy_version_id: universally unique identifier, policy_bundle: JSON format, benchmark_report: JSON format, training_metadata: JSON format). Optimization methods include the empirical distiller, the preference optimizer (Direct Preference Optimization DPO / Human Feedback-Based Reinforcement Learning RLHF), the reinforcement learning trainer (RLTrainer (Proximal Policy Optimization PPO), and the population evolver (Population Evolver). These algorithms are existing technologies and will not be elaborated upon, but in this invention, they are encapsulated as a gated offline training pipeline.

[0074] In computer system operation and maintenance practices, configuration parameter updates often face a contradiction between "real-time" and "stability": if all changes take effect immediately, any defects in the patch will directly affect online services; if all changes require lengthy offline verification, the system cannot respond to business changes in a timely manner. This step draws on the ideas of copy-on-write and hot patching in operating systems, dividing the update channel into a fast adaptation layer and a slow sedimentation layer. The fast adaptation layer only modifies lightweight configurations (such as prompts and route weights), similar to modifying application configuration files, with controllable risks and rapid effects; the slow sedimentation layer handles heavyweight changes (such as model weights and workflow topology), similar to operating system kernel upgrades, requiring thorough regression testing. In addition, different types of parameters naturally have different update frequencies and impact ranges, so seven dedicated optimizers are set up to handle Prompt, Few-Shot, tool routing, workflow, memory, evaluator, and model weights respectively, avoiding the resource waste and coupling risks caused by a one-size-fits-all update. By employing a dual-timescale scheduling mechanism and seven types of optimizers in parallel, the system can respond to business changes at the minute level through a rapid adaptation layer (such as Prompt optimization) and accumulate high-quality strategies at the hour / day level through a slow accumulation layer (such as model weight fine-tuning), thus avoiding system oscillations caused by blind online updates.

[0075] Furthermore, based on the above embodiments, the steps of verifying and releasing the optimized version of the policy through a multi-stage security governance pipeline, and deploying the verified version to the online environment, can be refined as follows:

[0076] The optimization results are verified through a multi-stage security governance pipeline to determine the policy version that passes offline evaluation. The multi-stage security governance pipeline includes version management, security rule scanning, sandbox simulation, and resource budget verification. When the identity and permission level of the operator change meet the conditions, online task requests are segmented according to a preset progressive traffic ratio, and the canary release of the policy version that passed offline evaluation is executed. The received canary deployment status and online indicator stream are compared and significance tested through an A / B testing engine, and the verified policy version is deployed to the online environment.

[0077] In this embodiment, the change operator refers to the user or system entity that submits the policy change request. Identity and permission levels refer to the operator's identity authentication and permission level. The preset progressive traffic ratio refers to the proportion of service traffic for the new policy version that is gradually increased during the canary release process, typically ranging from 1% to 5% to 20% to 100%. The A / B testing engine is a module used to compare the differences between the old and new policy versions on key business metrics and to assist in decision-making through statistical significance testing (p-value calculation). The canary deployment status includes information such as the currently deployed policy version identifier, traffic ratio, and deployment status. The online metric stream refers to metrics used to evaluate the online policy version, such as success rate, 95th percentile latency (latency_p95), cost per task (cost_per_task), and violation count (violation_count).

[0078] Specifically, the optimization results (candidate policy versions / quick-adaptation patches) are verified through a multi-stage security governance pipeline. For example, firstly, version management is used to perform version registration and dependency resolution (processing) on ​​the candidate policy versions / quick-adaptation patches, outputting version metadata frames (fields: policy_version_id: universally unique identifier, parent_version: universally unique identifier, dependencies: string array, release_notes: string); version management is existing technology and will not be elaborated further. Next, security rules are scanned on the candidate policy versions, outputting a security scan report (fields: policy_version_id: universally unique identifier, rule_violations: string array, risk_score: floating-point type, approval_status: enumeration type); the configuration of the security rule base is existing technology and will not be elaborated further. Candidate policy versions / rapid-adaptation patches obtained through security scanning are loaded into the isolated execution environment. Historical task trajectories are replayed (processing), and a simulation evaluation report is output (fields: policy_version_id: UUID, simulation_pass_rate: floating-point, regression_tests: array of objects, resource_usage: JSON format). Sandbox environment construction is existing technology and will not be elaborated upon. The system receives resource usage from the simulation evaluation report (input), performs execution cost estimation (processing), and outputs a budget approval frame (fields: policy_version_id: universally unique identifier, estimated_cost: floating-point, budget_approval: boolean). Cost calculation is existing technology and will not be elaborated upon. The final output is the policy version package that passes the offline evaluation (fields: policy version identifier, policy bundle, simulation pass rate, risk score, budget approval), which is written to the release candidate queue (such as the database of the version registry).

[0079] In software engineering, deploying untested code directly to the production environment is widely recognized as a high-risk operation. Similarly, if agent policy updates skip the verification process and go live directly, it may lead to sudden changes in system behavior, compliance violations, or resource exhaustion. This step mitigates these risks through a three-layered pre-protection mechanism: Safety Guardrails acts as a static code analysis tool, discovering potential security vulnerabilities and compliance risks during compilation; the sandbox simulator acts as an integration testing environment, replaying real-world scenarios to verify the backward compatibility of the new policy and ensure that it does not break existing functionality; and the Budget Controller manages resource quotas to prevent API costs from spiraling out of control due to logical flaws in the new policy (such as getting stuck in an infinite loop). These three layers work in tandem, forming a layered defense system similar to a network protocol stack. Through offline security scanning and sandbox simulation, unauthorized configurations and performance regressions are intercepted before policy deployment, reducing production environment failure rates and keeping security risks out of the isolated environment.

[0080] Specifically, the system receives policy versions that have passed offline evaluation, checks the operational permissions of the currently logged-in account of the operator making the change, and proceeds with the release only when the operator's identity and permission level meet the requirements, ensuring that only entities with administrator privileges can submit changes, and outputs a permission verification pass frame. For the permission verification pass frame and the verified policy package, a comprehensive approval process is performed, outputting an access control decision frame (fields: decision: enumeration type [pass / review_fail / approve], reviewer_comment: string); automated access control combined with manual review is existing technology and will not be elaborated further. For policy packages that pass access control, online task requests are routed to the new policy version according to a preset progressive traffic ratio, and the canary deployment status is output (fields: policy_version_id: universally unique identifier, traffic_percentage: floating-point, deployment_status: enumeration type); canary deployment technology is existing technology and will not be elaborated further. The A / B testing engine is used to compare and perform significance tests on the gray-scale deployment status and the online metrics stream (the A / B testing engine compares the differences between the old and new versions in task success rate, latency, cost, and security violation rate, and calculates the p-value). An A / B decision frame is output (fields: policy_version_id: a universally unique identifier, metric_delta: JSON format, p_value: floating-point, recommendation: enumeration type). The statistical testing method is existing technology and will not be elaborated. The validated policy version (fields: policy_version_id, traffic_percentage, deployment_status) is deployed to the online environment.

[0081] Even if a strategy performs well in offline testing, hidden flaws may still be exposed in the real online environment due to differences in data distribution, concurrency pressure, or changes in user behavior. Therefore, this step introduces the concept of canary release: by exposing the new strategy to only a small number of users (e.g., 1%), its performance under real traffic is observed. The A / B testing engine quantifies the differences between the old and new versions through statistical significance testing (p-value calculation), avoiding misjudgments caused by subjective judgment. Access control is used to prevent abuse or misoperation of permissions, ensuring that those initiating strategy changes have the appropriate permissions, similar to DBA access control in a database management system. This is an important technical means to ensure system governance compliance. Through gradual traffic switching, statistical significance testing, and access control mechanisms, the impact of strategy updates is controlled, avoiding systemic risks caused by full releases, and ensuring that all changes are traceable to the responsible party.

[0082] As a first optional embodiment of this embodiment, before evaluating the value of the verified structured execution trajectories to select high-value trajectories, compressing the execution experience of high-value trajectories, and transforming task-level execution experience into reusable skill templates at the task family level and storing them in a hierarchical memory bank, the following is also included:

[0083] Receive abnormal signals from external business metrics monitored by external authorities, mark the task families associated with the abnormal signals as high priority, accelerate the processing frequency of experience corresponding to high priority task families entering the learning layer, and trigger a re-evaluation and strategy re-optimization process for high priority task families.

[0084] In this embodiment, external monitoring refers to a business monitoring system independent of the intelligent agent system. A high-priority marker is an attribute marker for a task family, indicating that the experience related to that task family needs to be processed first.

[0085] Specifically, it receives KPI anomaly alarms (such as declining conversion rates and SLA breaches) from external monitoring systems, determines whether to trigger the external signal loop (KPI, technical indicator changes, and user ratings) relearning mechanism; if the threshold is exceeded, it generates an external business indicator anomaly signal, marks the task family as high priority, triggers the re-evaluation and strategy re-optimization process, and accelerates the processing frequency of entering the learning layer.

[0086] Relying solely on internal metrics for evolution can easily lead to a disconnect between the system and real business objectives. Therefore, anomaly signals from external business metrics (such as conversion rate and SLA achievement rate) are introduced as grounding anchors. When significant anomalies occur at the business level, the retraining process of the learning layer is proactively triggered, ensuring that the system's optimization direction remains consistent with real external needs. Through multi-source feedback fusion and an external signal loop mechanism, a true business anchor is established, preventing the system from learning incorrectly based on false or internally self-consistent feedback, thus improving the reliability and business relevance of the meta-learning data source.

[0087] As a second optional embodiment of this first embodiment, after verifying the credibility of the execution result based on the feedback signal and generating a verified structured execution trajectory, the method further includes:

[0088] For execution traces and error logs that fail the trustworthiness verification, perform failure mode analysis, generate failure mode records, and store them in the failure mode library of the hierarchical memory.

[0089] In this embodiment, an execution trajectory that fails the reliability verification refers to an execution trajectory that fails all verification levels after being verified by the hierarchical verification pipeline (Verifier). The error log is a collection of exception information recorded in the execution trajectory during the task execution process. The failure mode record is a structured output of failure mode analysis, containing fields such as mode identifier, cause vector, affected task family, and anti-pattern marker. The failure mode library is a dedicated storage component in the hierarchical memory used for persistently storing failure mode records.

[0090] Specifically, failure mode analysis is performed on the unverified trajectories and related error logs, and failure mode records are output (fields: pattern_id: UUID, cause_vector (JSON format), affected_tasks array: string array, anti_pattern_flag (boolean)). The failure mode records are stored in the failure mode library in the hierarchical memory for reverse defense, and the failure experience is abstracted into structured information.

[0091] As a third optional embodiment of this first embodiment, based on the above embodiments, it further includes:

[0092] In the execution planning phase of subsequent tasks, a reverse defense retrieval is performed based on the failure mode records in the failure mode library to obtain the reverse defense retrieval results. If the current task is determined to meet the triggering conditions of any failure mode based on the reverse defense retrieval results, the execution path corresponding to the failure mode is actively avoided or the execution step order is adjusted.

[0093] In this embodiment, the execution planning phase of subsequent tasks refers to the stage in which the system, after receiving a new task request, performs task parsing, plan generation, and tool invocation at the execution layer. The reverse defense retrieval result is the output of a search of the failure mode library, including matching failure mode records, matching similarity scores, and suggested avoidance strategies. The triggering condition is a set of conditional expressions defined in the failure mode record, used to determine whether the context of the current task is related to that failure mode. The execution path corresponding to the failure mode refers to the sequence of steps and tool invocation chain in the execution plan.

[0094] Specifically, during the memory retrieval process in the planning phase of subsequent tasks, in addition to searching the successful experience templates in the skill base, a failure mode library is also searched to check whether the contextual features of the current task match the triggering conditions of known failure modes. The retrieval output is a reverse defense retrieval result, including a list of matched failure mode records, a similarity score, and suggested avoidance strategies. When it is detected that the current task meets the triggering conditions of a certain failure mode, it does not execute according to the original plan, but actively adopts the execution path corresponding to the failure mode or adjusts the order of execution steps to avoid repeated failures. For example, avoidance methods include: skipping risky steps, adjusting the order of steps, using alternative tools, adding verification steps, lowering the execution timeout threshold, and triggering manual review in advance.

[0095] Through the reverse defense mechanism of the failure mode library, it is possible to identify and avoid similar error triggers in advance, rather than repeating trial and error.

[0096] For example, the content of this invention can be demonstrated using a specific system architecture. Figure 2 This is a schematic diagram of the architecture of a meta-learning method for an intelligent agent provided in Embodiment 1 of the present invention, as shown below. Figure 2The diagram illustrates the four-layer structure of the system: Execution Layer, Learning Layer, Governance Layer, and Data & Knowledge Layer, along with the collaborative relationships between their internal functional modules. The Execution Layer is the front line of the system. The API gateway receives HTTP / HTTPS requests; the task parsing module parses, schedules, and monitors tasks; the Multi-Model Orchestrator selects the appropriate LLM model based on task complexity; the Planner generates an execution plan based on retrieved skill templates; the Executor is responsible for scheduling execution; the Tool Manager calls external tools via HTTP / HTTPS; the Reflector module dynamically corrects errors when tasks fail; and the Trace Store module persists the execution trajectory. The Learning Layer is the core of meta-learning. The feedback system integrates five types of feedback signals: environment, rule, result, process, and preference. The verifier filters out reliable trajectories through a multi-layered verification pipeline and marks unlearnable trajectories. The observability agent receives external KPI anomaly signals and marks high-priority task families. The evaluator calculates the overall score and learning value, filtering high-value trajectories. The experience buffer caches high-value experiences and supports priority ranking. The skill miner performs a four-step compression process—attribution, abstraction, templating, and registration—to generate reusable skill templates. The failure analyzer writes failure experiences into a failure mode library for reverse defense. The policy optimizer matrix contains seven types of optimizers, distinguishing between fast adaptation layers and slow sedimentation layers through dual-timescale scheduling. The governance layer is responsible for secure deployment.The Version Registry manages policy versions and dependencies; the Safety Guardrails module performs security rule scans, outputting risk scores and approval status; the Sandbox Simulator replays historical trajectories in an isolated environment for verification; the Budget Controller reviews tokens and cost budgets; the Permission Isolator verifies operator permissions based on RBAC; the Access Control System outputs pass, fail, or approve decisions; the Canary Release module segments traffic according to a progressive traffic ratio; and the Rollback & Audit Logger module implements automatic rollback and tamper-proof auditing. The Data & Knowledge Layer provides unified storage. A vector database supports semantic retrieval of skill and failure mode libraries; a relational database stores version metadata and audit logs; a distributed time-series storage stores execution trajectories and observation metrics; and a unified data access interface provides standardized read and write services to all upper-layer modules.

[0097] Example 2

[0098] Figure 3 This is a schematic diagram of the structure of a meta-learning device for an intelligent agent provided in Embodiment 2 of the present invention. Figure 3 As shown, the device includes:

[0099] The signal determination module 31 is used to drive the agent to perform tasks based on the received task request and the hierarchical memory bank, and to collect feedback signals from multiple heterogeneous information sources during the execution process.

[0100] The trajectory generation module 32 is used to verify the credibility of the execution result based on the feedback signal and generate a verified structured execution trajectory.

[0101] The template storage module 33 is used to evaluate the value of the verified structured execution trajectory to select high-value trajectories, compress the execution experience of the high-value trajectories, and transform the task-level execution experience into reusable skill templates at the task family level, and store them in the hierarchical memory bank.

[0102] The strategy optimization module 34 is used to allocate the strategy optimization task to the fast adaptation layer or the slow accumulation layer for differentiated updates based on the reusable skill templates accumulated in the hierarchical memory bank, through a dual time scale scheduling mechanism, and generate optimization results.

[0103] The result determination module 35 is used to verify and release the optimized results through a multi-stage security governance pipeline, and deploy the verified policy version to the online environment.

[0104] The technical solution of this invention significantly improves the reliability and quality of execution data through a layered memory and multi-source feedback verification, providing a reliable foundation for subsequent learning. Through value assessment and experience compression, fragmented task experience is transformed into reusable task family skill templates, greatly increasing the cross-task experience reuse rate and reducing storage overhead. A dual-timescale scheduling mechanism enables a rapid adaptation layer to respond to business changes at the minute level and a slow accumulation layer to deeply optimize strategies, avoiding system instability caused by online updates. A multi-stage security governance pipeline ensures that strategy updates are fully verified and released incrementally, compressing fault recovery time to the minute level and significantly improving the stability and traceability of system deployment.

[0105] Furthermore, the signal determination module 31 is specifically used for:

[0106] Based on the task family identifier in the task request, retrieve the historical planning template that matches the task family identifier from the hierarchical memory and determine the retrieval result;

[0107] Based on the confidence level of the search results and the historical planning template, the target planning process is determined;

[0108] An initial execution plan is generated based on the target planning process, driving the agent to execute tasks in the order of the steps in the initial execution plan, and selecting the corresponding computing power model instance according to the task complexity during the execution process;

[0109] During execution, when a step fails or the result deviates from the expected result, the subsequent execution steps are dynamically corrected according to the replanning mechanism, the corrected execution path is determined and the correction notes are recorded.

[0110] The task continues to be executed based on the revised steps until the task is completed. The process feedback signals in the execution trajectory, the environmental feedback signals in the environmental return messages, the rule feedback signals output by the rule engine, and the result feedback signals and preference feedback signals of the user interaction interface are extracted.

[0111] Furthermore, the trajectory generation module 32 is specifically used for:

[0112] The feedback signals are weighted by credibility to generate structured feedback records;

[0113] The execution results and the structured feedback records are verified sequentially through a layered verification pipeline, which includes: preset logic rule verification, unit test verification, external fact-checking, simulator environment interaction verification, multi-review voting verification, and manual review verification.

[0114] At each verification level, if the execution result fails the verification of the current level, the execution trajectory that fails the verification is marked as unlearnable and prevented from entering the subsequent experience compression stage.

[0115] Otherwise, the execution trajectory that passes all verification levels will be marked as a trusted trajectory, and a structured execution trajectory will be determined.

[0116] Furthermore, the template storage module 33 is specifically used for:

[0117] The validated structured execution trajectories are evaluated for value to identify high-value trajectories;

[0118] Attribution analysis was performed on the high-value trajectories to locate key decision points and obtain attribution results;

[0119] The task processing pattern is abstracted from the attribution results to generate reusable skill templates;

[0120] The skill templates are registered to the hierarchical memory bank based on the conflict resolution mechanism.

[0121] Furthermore, the strategy optimization module 34 is specifically used for:

[0122] Based on a dual-time-scale scheduling mechanism, combined with the type and urgency of the policy update target and the current system load, a differentiated update layer is triggered, wherein the urgency is determined based on abnormal signals of external business indicators.

[0123] When the rapid adaptation layer is triggered, the online lightweight optimizer is invoked to generate a configuration-level patch as the optimization result.

[0124] When the slow sedimentation layer is triggered, the offline heavy optimizer is invoked to aggregate multiple skill templates accumulated in the hierarchical memory to generate candidate strategy versions, which are then used as the optimization results.

[0125] Furthermore, the result determination module 35 is specifically used for:

[0126] The optimization results are verified through a multi-stage security governance pipeline to determine the policy version that passes offline evaluation. The multi-stage security governance pipeline includes version management, security rule scanning, sandbox simulation, and resource budget verification.

[0127] When the operator's identity and permission level change meet the conditions, online task requests are segmented according to a preset progressive traffic ratio, and the gray release of the strategy version that has passed offline evaluation is executed.

[0128] The received gray-scale deployment status is compared and significance is verified by using an A / B testing engine, and the verified strategy version is then deployed to the online environment.

[0129] Optionally, the device may also include an external optimization module.

[0130] The external optimization module is specifically used to: before performing value evaluation on the verified structured execution trajectory to filter out high-value trajectories, compress the execution experience of the high-value trajectories, transform the task-level execution experience into a reusable skill template at the task family level, and store it in the hierarchical memory, receive an abnormal signal of external business indicators from external monitoring, and mark the task family associated with the abnormal signal of external business indicators as high priority.

[0131] Accelerate the processing frequency of experience entering the learning layer for the high-priority task families, and trigger a re-evaluation and strategy re-optimization process for the high-priority task families.

[0132] Optionally, the device further includes: a failure storage module, used to perform failure mode analysis on the execution trajectory and its error log that failed the reliability verification after the execution result is verified according to the feedback signal and a verified structured execution trajectory is generated, generate failure mode records and store them in the failure mode library of the hierarchical memory.

[0133] Furthermore, the failure storage module is specifically used for:

[0134] In the execution planning phase of subsequent tasks, a reverse defense retrieval is performed based on the failure mode records in the failure mode library to obtain the reverse defense retrieval results.

[0135] If the current task meets the triggering conditions of any failure mode based on the reverse defense retrieval results, then the execution path corresponding to the failure mode is actively avoided or the execution step order is adjusted.

[0136] The meta-learning device for intelligent agents provided in the embodiments of the present invention can execute the meta-learning method for intelligent agents provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0137] Example 3

[0138] Figure 4A schematic diagram of an electronic device 40 that can be used to implement embodiments of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0139] like Figure 4 As shown, the electronic device 40 includes at least one processor 41 and a memory, such as a read-only memory (ROM) 42 and a random access memory (RAM) 43, communicatively connected to the at least one processor 41. The memory stores computer programs executable by the at least one processor. The processor 41 can perform various appropriate actions and processes based on the computer program stored in the ROM 42 or loaded from storage unit 48 into the RAM 43. The RAM 43 can also store various programs and data required for the operation of the electronic device 40. The processor 41, ROM 42, and RAM 43 are interconnected via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0140] Multiple components in electronic device 40 are connected to I / O interface 45, including: input unit 46, such as keyboard, mouse, etc.; output unit 47, such as various types of monitors, speakers, etc.; storage unit 48, such as disk, optical disk, etc.; and communication unit 49, such as network card, modem, wireless transceiver, etc. Communication unit 49 allows electronic device 40 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0141] Processor 41 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 41 performs the various methods and processes described above, such as meta-learning methods for intelligent agents.

[0142] In some embodiments, the meta-learning method of the agent can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 40 via ROM 42 and / or communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the meta-learning method of the agent described above can be performed. Alternatively, in other embodiments, processor 41 can be configured to execute the meta-learning method of the agent by any other suitable means (e.g., by means of firmware).

[0143] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0144] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0145] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0146] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0147] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0148] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0149] In one embodiment, the present invention further includes a computer program product, which includes a computer program that, when executed by a processor, implements the meta-learning method of the intelligent agent according to any embodiment of the present invention.

[0150] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0151] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0152] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A meta-learning method for an intelligent agent, characterized in that, include: Based on the received task request and the hierarchical memory bank, the agent executes the task and collects feedback signals from multiple heterogeneous information sources during the execution process. The credibility of the execution result is verified based on the feedback signal, and a verified structured execution trajectory is generated. The validated structured execution trajectories are evaluated for value to select high-value trajectories, and the execution experience of the high-value trajectories is compressed to transform the task-level execution experience into reusable skill templates at the task family level, which are then stored in the hierarchical memory bank. Based on the reusable skill templates accumulated in the hierarchical memory bank, the strategy optimization task is assigned to the fast adaptation layer or the slow sedimentation layer for differentiated updates through a dual time-scale scheduling mechanism to generate optimization results. Based on the optimization results, the policy is verified and released through a multi-stage security governance pipeline, and the verified policy version is deployed to the online environment.

2. The method according to claim 1, characterized in that, The process involves the agent executing tasks based on received task requests and a hierarchical memory, and collecting feedback signals from multiple heterogeneous information sources during execution, including: Based on the task family identifier in the task request, retrieve the historical planning template that matches the task family identifier from the hierarchical memory and determine the retrieval result; Based on the confidence level of the search results and the historical planning template, the target planning process is determined; An initial execution plan is generated based on the target planning process, driving the agent to execute tasks in the order of the steps in the initial execution plan, and selecting the corresponding computing power model instance according to the task complexity during the execution process; During execution, when a step fails or the result deviates from the expected result, the subsequent execution steps are dynamically corrected according to the replanning mechanism, the corrected execution path is determined and the correction notes are recorded. The task continues to be executed based on the revised steps until the task is completed. The process feedback signals in the execution trajectory, the environmental feedback signals in the environmental return messages, the rule feedback signals output by the rule engine, and the result feedback signals and preference feedback signals of the user interaction interface are extracted.

3. The method according to claim 1, characterized in that, The step of verifying the credibility of the execution result based on the feedback signal and generating a verified structured execution trajectory includes: The feedback signals are weighted by credibility to generate structured feedback records; The execution results and the structured feedback records are verified sequentially through a layered verification pipeline, which includes: preset logic rule verification, unit test verification, external fact-checking, simulator environment interaction verification, multi-review voting verification, and manual review verification. At each verification level, if the execution result fails the verification of the current level, the execution trajectory that fails the verification is marked as unlearnable and prevented from entering the subsequent experience compression stage. Otherwise, the execution trajectory that passes all verification levels will be marked as a trusted trajectory, and a structured execution trajectory will be determined.

4. The method according to claim 1, characterized in that, Before performing value assessment on the validated structured execution trajectories to filter out high-value trajectories, compressing the execution experience of the high-value trajectories to transform task-level execution experience into reusable skill templates at the task family level, and storing them in the hierarchical memory, the method further includes: Receive abnormal signals of external business indicators from external monitoring, and mark the task family associated with the abnormal signals of external business indicators as high priority; Accelerate the processing frequency of experience entering the learning layer for the high-priority task families, and trigger a re-evaluation and strategy re-optimization process for the high-priority task families.

5. The method according to claim 1, characterized in that, The process of evaluating the value of the validated structured execution trajectories to select high-value trajectories, compressing the execution experience of the high-value trajectories to transform task-level execution experience into reusable skill templates at the task family level, and storing them in the hierarchical memory bank includes: The validated structured execution trajectories are evaluated for value to identify high-value trajectories; Attribution analysis was performed on the high-value trajectories to locate key decision points and obtain attribution results; The task processing pattern is abstracted from the attribution results to generate reusable skill templates; The skill templates are registered to the hierarchical memory bank based on the conflict resolution mechanism.

6. The method according to claim 1, characterized in that, After verifying the credibility of the execution result based on the feedback signal and generating a verified structured execution trajectory, the method further includes: For execution trajectories and their error logs that fail the credibility verification, perform failure mode analysis, generate failure mode records, and store them in the failure mode library of the hierarchical memory.

7. The method according to claim 6, characterized in that, Also includes: In the execution planning phase of subsequent tasks, a reverse defense retrieval is performed based on the failure mode records in the failure mode library to obtain the reverse defense retrieval results. If the current task meets the triggering conditions of any failure mode based on the reverse defense retrieval results, then the execution path corresponding to the failure mode is actively avoided or the execution step order is adjusted.

8. The method according to claim 1, characterized in that, The process involves allocating strategy optimization tasks to either a fast adaptation layer or a slow accumulation layer for differentiated updates based on the reusable skill templates accumulated in the hierarchical memory, using a dual-time-scale scheduling mechanism, to generate optimization results, including: Based on a dual-time-scale scheduling mechanism, combined with the type and urgency of the policy update target and the current system load, a differentiated update layer is triggered, wherein the urgency is determined based on abnormal signals of external business indicators. When the rapid adaptation layer is triggered, the online lightweight optimizer is invoked to generate a configuration-level patch as the optimization result. When the slow sedimentation layer is triggered, the offline heavy optimizer is invoked to aggregate multiple skill templates accumulated in the hierarchical memory to generate candidate strategy versions, which are then used as the optimization results.

9. The method according to claim 1, characterized in that, Based on the optimization results, the process involves verification and release through a multi-stage security governance pipeline, and the deployment of the verified policy version to the online environment, including: The optimization results are verified through a multi-stage security governance pipeline to determine the policy version that passes offline evaluation. The multi-stage security governance pipeline includes version management, security rule scanning, sandbox simulation, and resource budget verification. When the operator's identity and permission level change meet the conditions, online task requests are segmented according to a preset progressive traffic ratio, and the gray release of the strategy version that has passed offline evaluation is executed. The received gray-scale deployment status is compared and significance is verified by using an A / B testing engine, and the verified strategy version is then deployed to the online environment.

10. A meta-learning device for an intelligent agent, characterized in that, include: The signal determination module is used to drive the agent to execute tasks based on the received task request and hierarchical memory, and to collect feedback signals from multiple heterogeneous information sources during the execution process. The trajectory generation module is used to verify the credibility of the execution result based on the feedback signal and generate a verified structured execution trajectory. The template storage module is used to evaluate the value of the verified structured execution trajectory to select high-value trajectories, compress the execution experience of the high-value trajectories, and transform the task-level execution experience into reusable skill templates at the task family level, which are then stored in the hierarchical memory bank. The strategy optimization module is used to allocate the strategy optimization task to the fast adaptation layer or the slow accumulation layer for differentiated updates based on the reusable skill templates accumulated in the hierarchical memory bank, and generate optimization results through a dual time scale scheduling mechanism. The result determination module is used to verify and release the optimization results through a multi-stage security governance pipeline, and deploy the verified policy version to the online environment.

11. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the meta-learning method of the agent according to any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the meta-learning method of the intelligent agent as described in any one of claims 1-9.

Images