An agent operation and maintenance system based on a self-evolution framework and an application method thereof

Abstract

The application discloses an intelligent agent operation and maintenance system based on a self-evolution framework and an application method thereof, relates to the technical field of artificial intelligence and intelligent agent operation and maintenance, and adopts a four-layer modular layered architecture of an intelligent agent application layer, a runtime support layer, an operation and maintenance management layer and an infrastructure layer from top to bottom, constructs a safe self-inspection driven "state modeling-skill combination-risk constraint-over matching interception-nonparametric experience evolution" closed loop, forms a self-evolution closed loop with short-term decision optimization and long-term capability evolution in parallel, and realizes the whole life cycle unmanned operation and maintenance and autonomous evolution of the intelligent agent. The application effectively solves the problems of lack of unified architecture in the intelligent agent operation and maintenance field, inability of the intelligent agent to safely and autonomously evolve, high operation and maintenance cost in the whole life cycle, uncontrollable decision safety and the like in the prior art.

Description

Technical Field

[0001] This invention relates to the technical field of artificial intelligence and intelligent agent operation and maintenance, and in particular to an intelligent agent operation and maintenance system based on a self-evolutionary framework and its application method. Background Technology

[0002] With the rapid development of large-scale models and intelligent agent technologies, intelligent agents with autonomous decision-making and execution capabilities have become the core carriers for the engineering application of artificial intelligence. Currently, existing technologies in the fields of autonomous driving intelligent agents, intelligent agent operation and maintenance, safety decision-making, and automation can be mainly divided into three categories: intelligent agent decision-making and safety constraint technologies for autonomous driving, autonomous driving decision-making technologies based on experience / case reuse, and MLOps / model lifecycle operation and maintenance technologies. Furthermore, existing reinforcement learning fields often employ the Actor-Critic architecture for safety decision-making, and related research largely focuses on the algorithm level, failing to integrate with engineering operation and maintenance, CI / CD, OTA, simulation testing, skill management, and other systems, thus unable to support automated operation and maintenance throughout the entire lifecycle of intelligent agents.

[0003] Furthermore, existing technologies suffer from the following systemic defects that severely restrict the large-scale deployment of intelligent agents: First, there is a lack of dedicated AgentOps architecture and a unified engineering foundation: Traditional MLOps (General Operations Framework) or DevOps (Extensions of General Operations Framework for Machine Learning Scenarios) primarily target offline training and online inference in data centers, failing to adapt to the characteristics of intelligent agents such as real-time decision-making, dynamic interaction, continuous evolution, and physical constraints. They also lack layered architectures, standardized components, and end-to-end orchestration capabilities for intelligent agents. Second, capabilities are statically fixed, with insufficient dynamic assembly and generalization capabilities: The intelligent agent's skill set is finalized during the development phase, unable to be dynamically combined or quickly adapted according to scenario and task requirements. It struggles with long-tail scenarios, exhibits weak cross-task transfer learning and continuous evolution capabilities, and is prone to getting stuck in local optima or catastrophic forgetting. Third, the entire lifecycle is highly dependent on manual intervention, resulting in high operational costs: The entire process of intelligent agent development, training, deployment, testing, iteration, and rollback requires manual intervention. It lacks automated CI / CD, quality gates, version control, and anomaly rollback mechanisms, leading to version inconsistencies, low iteration efficiency, and high long-term maintenance costs. Fourth, there is a severe imbalance between evolution and security, and a lack of unified security constraints: existing self-evolutionary algorithms mostly optimize performance metrics alone, lacking real-time risk verification, decision interception, and security introspection mechanisms. Policy updates are prone to introducing unknown security risks, failing to guarantee the security and stability of the agent's continuous evolution. Fifth, experience cannot be accumulated and reused, resulting in low evolutionary efficiency and a high risk of drift: decision logs are stored in a fragmented manner, making it difficult to transform them into reusable structured experience. Capability iteration relies on retraining the model, which easily leads to model parameter drift, decreased system stability, long iteration cycles, and high resource consumption.

[0004] To address the aforementioned shortcomings, a general, secure, and self-evolving intelligent agent operation and maintenance system is needed to provide a unified, engineered solution for various complex intelligent agents. Summary of the Invention

[0005] This invention provides an intelligent agent operation and maintenance system and its application method based on a self-evolutionary framework, which solves the problems of lack of unified architecture, inability of intelligent agents to evolve securely, high operation and maintenance costs throughout the entire life cycle, and uncontrollable decision security in the field of intelligent agent operation and maintenance.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: In the first aspect, the present invention provides an intelligent agent operation and maintenance system based on a self-evolutionary framework. The system adopts a four-layer modular layered architecture from top to bottom, consisting of an intelligent agent application layer, a runtime support layer, an operation and maintenance management layer, and an infrastructure layer. Each layer realizes data interaction, instruction flow, and status feedback through standardized APIs and event-driven mechanisms, thereby realizing a closed-loop decision-making mechanism for unmanned operation and maintenance and autonomous evolution of the intelligent agent throughout its entire life cycle. The agent application layer is used to deploy target agent instances that directly interact with the target scenario; the runtime support layer includes a skill library, a dynamic assembly engine, a security introspection module, a policy service module, and a state manager, which are used to provide a standardized operating environment and core functional services for the target agents deployed in the agent application layer. The core functional services include encapsulating underlying resource differences, dynamically assembling skills, and real-time interception of insecure decisions; the operation and maintenance management layer is used to realize the full lifecycle automated management of the target agents; the infrastructure layer is used to provide physical and virtual resource support for the operation and maintenance system. The construction of the closed-loop decision-making mechanism includes: The state modeling unit is used to perform unified modeling of the agent's operating scenario based on the target agent and the target scenario, obtain the agent's current global structured state vector, and retrieve a set of candidate skills in the skill base based on the agent's current global structured state vector through a similarity function; The skill combination unit is used to abstract each agent skill of the target agent into a structured quintuple, and to combine the candidate skills in the candidate skill set to generate candidate control actions. The risk constraint unit is used to perform a safety assessment of the candidate control actions based on a decision constraint mechanism using a risk function. The security interception unit is used to trigger the security interception mechanism when the risk score value of the security assessment exceeds a preset threshold, dynamically expand the security constraints and generate the least risky security replacement action; An experience evolution unit is used to, when the agent is in a new scenario state with its current global structured state vector, adopt a non-parametric experience transfer mechanism based on the candidate decisions already executed, and update the skills in the skill library that best match the candidate decisions for the new scenario state to obtain the candidate control actions corresponding to the new scenario state; the candidate decisions are either the candidate control actions or the safety replacement actions.

[0007] In one possible implementation, the experience evolution unit is specifically configured to perform: After the candidate decision is executed, the current decision-making process is encapsulated as an experience tuple and stored in a distributed experience repository; the distributed experience repository includes multiple historical experience tuples encapsulated from historical decision-making processes. For a new scene state in the current global structured state vector of the agent, the most matching historical experience is obtained from the distributed experience base through similarity retrieval; The Frenet coordinate system trajectory translation and scaling algorithm is used to perform scene adaptive correction between the new scene state and the best matching historical experience, so as to determine the candidate control action corresponding to the new scene state based on the skill executed according to the best matching historical experience.

[0008] In one possible implementation, the experience evolution unit is further configured to perform: High-value experience tuples are periodically extracted from the distributed experience base, new composite skills are generated using clustering algorithms, and composite skills verified through automated simulation testing are added to the skill base to update the skills in the skill base; the high-value experience tuples refer to experience tuples with a success rate greater than a first preset value and a complexity greater than a second preset value.

[0009] In one possible implementation, the state modeling unit is specifically configured to perform: A unified model of the agent's operating scenario is performed based on the target agent and the target scenario. The multimodal perception data of the agent's operating scenario is mapped into the agent's current global structured state vector to describe the current environment and task constraints. The agent's current global structured state vector includes the agent's own position state, the agent's own velocity state, the position state of the entities surrounding the agent, the velocity state of the entities surrounding the agent, the environmental topology, the set of rule constraints, and the set of objects perceived by the agent. Using the agent's current global structured state vector as input, a weighted cosine similarity function is used to compare the agent's historical scene states, and the most similar target historical scene states are selected through a Top-K filtering operation. These are then mapped to the skill library to obtain a set of candidate skills corresponding to the target historical scene states.

[0010] In one possible implementation, the skill combination unit includes components specifically configured to perform: Each agent skill of the target agent is abstracted into a structured quintuple; the structured quintuple of the agent skill includes the skill application conditions, the skill execution pre-constraints, the skill control parameter space, the policy mapping function, and the skill execution effect evaluation index. Based on the first calculation formula, the candidate control action corresponding to the weighted fusion of all candidate skills in the candidate skill set is determined; the first calculation formula is as follows: ; in, Indicates candidate control actions; The current control cycle represents the timing of the agent's control. This represents the total number of candidate skills in the candidate skill set. Indicates the current candidate skill number; This represents the weight coefficient of the current candidate skill. Indicates the first The policy mapping function for each candidate skill. express The current global structured state vector of the agent at time step.

[0011] In one possible implementation, the skill combination unit also includes a pre-set multi-skill conflict resolution mechanism; When there is a conflict in the control instructions of multiple candidate skills in the candidate skill set, the skill combination unit triggers a multi-skill conflict resolution mechanism and generates candidate control actions by using a priority sorting and constraint fusion method; the priority of the candidate skills in the candidate skill set is from high to low as safety skills, task skills and comfort skills.

[0012] In one possible implementation, the risk constraint unit is specifically configured to perform: A decision constraint mechanism based on a risk function is used to assess the safety of the candidate control actions and obtain a risk score; the risk function is specifically: ; in, express Time-based candidate control actions The corresponding risk score, This represents the total number of target entities present within the current sensing range. Indicates the target agent and the first The relative distance between target entities This represents the remaining time before the target agent collides with the i-th target entity. This represents the speed of the target intelligent agent. This indicates the speed of the target entity. It is a very small positive number. , , These are the weighting parameters for distance risk, collision time risk, and relative speed risk, respectively.

[0013] In one possible implementation, the security interception mechanism is an over-match rejection mechanism; the security interception unit is specifically configured to execute: When the risk score is less than or equal to a preset threshold, the candidate control action is executed; When the risk score is greater than the preset threshold, the overmatch rejection mechanism is triggered, the safety distance constraint is actively expanded and the decision problem is reconstructed into a constraint optimization problem, and the safe replacement action with the lowest risk is generated.

[0014] In one possible implementation, when actively extending the safety distance constraint and reconstructing the decision problem into a constraint optimization problem to generate the safest alternative action with the least risk, the security interception unit is specifically configured to execute: The safety distance constraint is dynamically expanded based on the risk score using a second calculation formula; the second calculation formula is specifically as follows: ; in, This indicates the expanded safety distance. Indicates the original safe distance; This represents the safety expansion coefficient, which is dynamically adjusted based on the risk score value. =1+0.5 ( θ), λ∈[1.2,2.0]; express Time-based candidate control actions The corresponding risk score, where θ represents the preset threshold; The decision-making process is restructured into a minimum risk optimization problem with extended safety constraints based on the third calculation formula, which is specifically: ; in, This represents the optimal safe replacement action obtained from the solution. This means finding the action with the least risk within the available action space. This indicates compliance with a set of safety constraints. This represents the expanded set of safety constraints.

[0015] Secondly, the present invention provides an application method for an intelligent agent operation and maintenance system based on a self-evolutionary framework. This method is based on the four-layer modular hierarchical architecture of the intelligent agent operation and maintenance system based on the self-evolutionary framework described in any of the above claims, and constructs a closed-loop decision-making mechanism to realize safe decision-making for autonomous driving and autonomous evolution of the intelligent agent.

[0016] The agent operation and maintenance system based on a self-evolutionary framework provided in this invention addresses the core pain points of existing agent operation and maintenance systems (AgentOps systems), such as lack of architecture, rigid skills, imbalance between security and evolution, and parameter drift risks. It deeply integrates the DevSecOps software development concept into the entire lifecycle management of agents, and adopts a four-layer modular architecture from top to bottom, consisting of agent application layer, runtime support layer, operation and maintenance management layer, and infrastructure layer. It constructs a closed loop driven by security introspection: "state modeling - skill combination - risk constraint - over-matching interception - non-parametric experience evolution", forming a self-evolutionary closed loop in parallel with short-term decision optimization and long-term capability evolution, realizing unmanned operation and maintenance and autonomous evolution of agents throughout their entire lifecycle.

[0017] The intelligent agent operation and maintenance system based on the self-evolutionary framework provided in this invention is applicable to the full life cycle management, capability evolution and security operation and maintenance of intelligent agents in various complex dynamic environments. The system has high versatility and can be widely applied to complex intelligent agent scenarios such as autonomous driving and software development. Attached Figure Description

[0018] Figure 1 A schematic diagram of a four-layer modular layered architecture for an intelligent agent operation and maintenance system based on a self-evolutionary framework, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram illustrating the construction of a loop decision-making mechanism in an intelligent agent operation and maintenance system based on a self-evolutionary framework, as provided in an embodiment of the present invention. Figure 3 This is a schematic diagram illustrating the implementation of an Actor-Critic dual-agent safety decision-making mechanism for intelligent driving decision-making in an intelligent agent operation and maintenance system based on a self-evolutionary framework, as provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of this disclosure, unless otherwise stated, "a plurality of" means two or more. Furthermore, the use of "based on" or "according to" implies openness and inclusiveness, because processes, steps, calculations, or other actions "based on" or "according to" one or more of the stated conditions or values ​​may in practice be based on additional conditions or beyond the stated values.

[0021] To address the issues of lack of a unified architecture, inability of intelligent agents to evolve securely, high maintenance costs throughout the entire lifecycle, and uncontrollable decision-making security in the field of intelligent agent operation and maintenance, this invention provides an intelligent agent operation and maintenance system and its application method based on a self-evolutionary framework.

[0022] like Figure 1 , Figure 2 As shown, this embodiment of the invention provides an intelligent agent operation and maintenance system based on a self-evolutionary framework. The system adopts a four-layer modular architecture from top to bottom, consisting of an intelligent agent application layer, a runtime support layer, an operation and maintenance management layer, and an infrastructure layer. Each layer realizes data interaction, instruction flow, and status feedback through standardized APIs and event-driven mechanisms, thereby achieving a closed-loop decision-making mechanism for unmanned operation and maintenance and autonomous evolution of the intelligent agent throughout its entire lifecycle.

[0023] In the four-layer modular architecture of this system, data interaction and command flow between layers are achieved through API interfaces and event-driven mechanisms. Components within a layer are decoupled, and functions between layers are coordinated, as detailed below: The agent application layer is used to deploy target agent instances that directly interact with the target scenario.

[0024] The runtime support layer includes a skill library, a dynamic assembly engine, a security introspection module, a policy service module, and a state manager, which provide a standardized operating environment and core functional services for the target intelligent agents deployed in the intelligent agent application layer.

[0025] The operations and maintenance management layer is used to achieve automated management of the entire lifecycle of the target intelligent agent.

[0026] The infrastructure layer is used to provide physical and virtual resource support for the operation and maintenance system.

[0027] Specifically, the infrastructure layer includes computing units, cloud servers, simulation clusters, data storage memory pools, and ROS2 communication middleware.

[0028] In the intelligent agent operation and maintenance system of this invention, when used to implement safety decisions for autonomous driving: the computing unit is an on-board computing unit, including an autonomous driving domain controller, sensor and actuator interfaces, and the on-board computing unit supports real-time model inference and vehicle control. A cloud server is used for large-scale model training and skill iteration. A simulation cluster is used to provide a high-fidelity, high-concurrency autonomous driving scenario simulation environment. The data storage memory pool is a dual-memory pool data storage module, including a local memory pool and a shared global memory pool; the data storage memory pool realizes the storage and reuse of driving logs, decision records, safety events, and cross-task knowledge. The ROS2 communication middleware provides standardized data interaction interfaces for each layer, namely the ROS2 communication interface.

[0029] The operations and maintenance management layer includes a CI / CD pipeline, a data playback and annotation module, a model training and distillation module, a simulation testing platform, a version control module, a quality access control module, and an OTA update module.

[0030] The CI / CD pipeline is used for the continuous integration and deployment of driving skills. The data playback and annotation module extracts typical scenarios from real-vehicle / simulation data for skill training. The model training and distillation module combines large language models, human driving data, and vehicle physics models to train lightweight driving skills suitable for in-vehicle edge deployment. The simulation testing platform performs automated testing of skills / models across all dimensions, including unit testing, integration testing, and safety testing. The version control module manages versions of agent code, models, configurations, and the skill library. The quality gate module presets performance thresholds such as collision rate, safety score, and comfort level, automatically blocking unqualified versions. The OTA update module enables remote deployment of skills / models and automatic rollback of abnormal versions.

[0031] The runtime support layer is the core execution and security layer of the intelligent agent operation and maintenance system. It provides a standardized operating environment and core functional services for the intelligent agent application layer. The core functional services include encapsulating underlying resource differences, dynamically assembling skills, and intercepting unsafe decisions in real time.

[0032] The runtime support layer specifically includes a skill library, a dynamic assembly engine, a security introspection module, a policy service module, and a state manager.

[0033] The system comprises several key components: a skill library for storing standardized basic and composite driving skills, supporting versioning and combined queries, and generalization capabilities across scenarios and traffic rules; a dynamic assembly engine for dynamically selecting and combining optimal skill sets from the skill library based on scenario perception results and driving task requirements, enabling on-demand assembly of capabilities by the autonomous driving agent; a safety introspection module as the core safety component, including a risk prediction network, an over-match rejection mechanism, and an Actor-Critic dual-agent introspection architecture, which calculates driving risk scores in real time, intercepts unsafe decision commands, and achieves a balance between driving performance and safety; a policy service module providing high-level decision-making logic, supporting both rule-driven and lightweight model-driven implementations; and a state manager continuously collecting agent decision logs, performance metrics, and anomalies and feeding them back to the operations and maintenance management layer, providing data support for skill iteration and model optimization.

[0034] The intelligent agent application layer is used to deploy target intelligent agent instances that directly interact with the target scenario. Each intelligent agent is assembled on demand by the runtime support layer, sharing the underlying services and skill library to avoid redundant development. This includes driving intelligent agents, diagnostic intelligent agents, and personalized intelligent agents.

[0035] The driving agent handles core autonomous driving tasks, covering the entire process of perception fusion, decision-making and planning, and control execution. The diagnostic agent monitors the vehicle's health status in real time, predicts faults, and triggers maintenance commands. The personalized agent learns the user's driving habits and dynamically adjusts the driving style.

[0036] The construction of a closed-loop decision-making mechanism includes: The state modeling unit is used to perform unified modeling of the agent's operating scenario based on the target agent and the target scenario, obtain the agent's current global structured state vector, and retrieve a set of candidate skills from the skill base based on the agent's current global structured state vector through a similarity function.

[0037] The skill combination unit is used to abstract each agent skill of the target agent into a structured quintuple, and to combine candidate skills in the candidate skill set to generate candidate control actions.

[0038] The risk constraint unit is used for a decision constraint mechanism based on a risk function to evaluate the safety of candidate control actions.

[0039] The security interception unit is used to trigger the security interception mechanism when the risk score value of the security assessment exceeds a preset threshold, dynamically expand the security constraints, and generate the least risky security replacement action.

[0040] The experience evolution unit is used to obtain the candidate control action corresponding to the new scenario state when the agent's current global structured state vector is in a new scenario state. Based on the candidate decisions that have been executed, a non-parametric experience transfer mechanism is adopted to update the skills in the skill library that best match the candidate decisions of the new scenario state. The candidate decisions are either candidate control actions or safe replacement actions.

[0041] The agent operation and maintenance system based on a self-evolutionary framework provided in this invention addresses the core pain points of existing agent operation and maintenance systems (AgentOps systems), such as lack of architecture, rigid skills, imbalance between security and evolution, and parameter drift risks. It deeply integrates the DevSecOps software development concept into the entire lifecycle management of agents, and adopts a four-layer modular architecture from top to bottom, consisting of agent application layer, runtime support layer, operation and maintenance management layer, and infrastructure layer. It constructs a closed loop driven by security introspection: "state modeling - skill combination - risk constraint - over-matching interception - non-parametric experience evolution", forming a self-evolutionary closed loop in parallel with short-term decision optimization and long-term capability evolution, realizing unmanned operation and maintenance and autonomous evolution of agents throughout their entire lifecycle.

[0042] Furthermore, the state modeling unit is specifically configured to execute: Based on the target agent and the target scenario, a unified model of the agent's operating scenario is performed. The multimodal perception data of the agent's operating scenario is mapped into the agent's current global structured state vector to describe the current environment and task constraints.

[0043] The agent's current global structured state vector includes the agent's own position state, the agent's own velocity state, the position state of the entities surrounding the agent, the velocity state of the entities surrounding the agent, the environmental topology, the set of rule constraints, and the set of objects perceived by the agent.

[0044] Specifically, the agent's current global structured state vector can be represented as: ; in, Represents the current global structured state vector of the agent, which is the set of the agent's current global states; It represents the agent's own position state as a position vector, such as the agent's two-dimensional or three-dimensional coordinates, pose, trajectory position, etc. Represents the agent's own velocity state, which is the agent's own velocity vector, such as: speed, heading, motion velocity vector, etc. Indicates the first The location status of entities surrounding an intelligent agent, which can be pedestrians, vehicles, obstacles, and other intelligent agents around the intelligent agent; Indicates the first The velocity states of entities surrounding an agent are used to characterize the motion characteristics of entities surrounding the agent. It represents the environmental topology, including static / semi-static structural information such as road network topology, scene layout, connectivity, road structure, and spatial topology relationships; This represents the set of rules and constraints, which includes all rules that must be followed, such as traffic rules, behavioral constraints, safety regulations, task constraints, ethical rules, and movement restrictions. This represents the set of objects perceived by the agent, including all targets, obstacles, environmental observation information, and perceived features perceived by the agent.

[0045] Using the agent's current global structured state vector as input, a weighted cosine similarity function is used to compare the agent's historical scene states. The most similar target historical scene states are then selected through a Top-K filtering operation and mapped to the skill base to obtain a set of candidate skills corresponding to the target historical scene states, thus realizing the retrieval and matching of scene states to behavioral skills.

[0046] Specifically, the selection process for candidate skills can be represented as follows: ; in, . This represents the weighted cosine similarity function, used to calculate the degree of similarity between the agent's current global structured state vector and the agent's historical scene states; Indicates the first Each intelligent agent's historical scenario state It corresponds to the complete scene state at a certain point in the past and binds the corresponding executable skill in the skill library. The filtering operation refers to sorting the results by similarity from highest to lowest and selecting the top K most similar matches. This is a set of candidate skills.

[0047] Furthermore, the skill set units include specific configurations for execution: Each agent skill of the target agent is abstracted into a structured quintuple. This structured quintuple includes the skill's applicable conditions, pre-execution constraints, the skill's control parameter space, the policy mapping function, and the skill's performance evaluation metrics.

[0048] Each agent skill of the target agent is abstracted as a structured quintuple, specifically represented as follows: ; in, A structured quintuple representing the skills of a single agent. This represents the applicable conditions (state space range) of a skill, used to limit the environment and the agent's own state range in which the skill can be triggered and allowed to take effect. It applies only to the agent's current global structured state vector. Falling This skill is only eligible to be activated within the defined state range. The pre-execution constraints refer to the hard boundary constraints that must be met before a skill can be started and executed. These are entry-level constraints, such as minimum safe following distance, maximum driving speed, road access rights, and lane boundary restrictions. If P is not met, the skill cannot be executed even in the applicable state I. The skill control parameter space refers to the range and domain of the output control quantity of the skill. It is the constraint boundary of the action output, such as the upper and lower limits of the steering angle, the range of acceleration / deceleration, the throttle opening range, the heading angle constraint, etc. The control parameters of the final output of the skill must fall within the C space. The policy mapping function refers to the core decision-making mechanism within the skill, which is implemented from the agent's current global structured state vector. The mapping to the original control command can take the form of structured rules (expert rules, logical judgments) or lightweight neural networks, with the input state and the output being the original control action quantity corresponding to the skill. The performance evaluation index for skill execution is a set of performance indicators used to quantitatively evaluate the quality of skill execution. Examples include: task completion success rate, average response time, driving safety score, ride comfort, energy consumption index, etc., which are used for skill optimization, weight updates, and performance iteration.

[0049] Based on the first calculation formula, the candidate control actions corresponding to the weighted fusion of all candidate skills in the candidate skill set are determined; the first calculation formula is as follows: ; in, Indicates candidate control actions; The current control cycle represents the timing of the agent's control. This represents the total number of candidate skills in the candidate skill set. Indicates the current candidate skill number; This represents the weight coefficient of the current candidate skill. Indicates the first The policy mapping function for each candidate skill. express The current global structured state vector of the agent at time step.

[0050] In this embodiment, The initial value is set to 0.6, which can be dynamically adjusted by the operations and maintenance management team.

[0051] Furthermore, the skill combination unit also has a pre-set multi-skill conflict resolution mechanism.

[0052] When there are conflicts in the control instructions of multiple candidate skills in the candidate skill set, the skill combination unit triggers a multi-skill conflict resolution mechanism, which generates candidate control actions by using priority sorting and constraint fusion.

[0053] The priority of candidate skills in the candidate skill set, from highest to lowest, is as follows: safety skills, task skills, and comfort skills. Safety skills have the highest priority and correspond to emergency avoidance, emergency braking, etc., which are mandatory constraints. Task skills have a medium priority and correspond to business task behaviors such as autonomous lane changing and path tracking. Comfort skills have the lowest priority and correspond to experience optimization behaviors such as smooth driving and gentle acceleration and deceleration.

[0054] By constraining the control parameters of low-priority skills, they can meet the prerequisites for high-priority skills.

[0055] Furthermore, the risk constraint unit is specifically configured to execute: A decision constraint mechanism based on a risk function is used to assess the safety of candidate control actions and obtain a risk score. The risk function is as follows: ; in, express Time-based candidate control actions The corresponding risk score value indicates that a higher risk score value indicates a better candidate control action. The higher the security risk, the better.

[0056] This represents the total number of target entities within the current sensing range, where i represents the current target entity number.

[0057] Indicates the target agent and the first The relative distance between target entities This represents the remaining time before the target agent collides with the i-th target entity. This represents the speed of the target intelligent agent. This represents the speed of the i-th target entity. To prevent extremely small positive numbers with a denominator of zero, , , These are the weighting parameters for distance risk, collision time risk, and relative speed risk, respectively.

[0058] This indicates the risk of being close to the source; the closer the distance, the greater the risk. The smaller, The larger the value, the higher the risk score.

[0059] Indicates the risk of collision time. The smaller the value, the more likely a collision is to occur. The larger the value, the higher the risk score.

[0060] Representing relative speed risk, the faster the target agent's speed is relative to the i-th target entity, the higher the risk score.

[0061] The aforementioned risk function enables a unified quantitative evaluation of candidate control actions, transforming the decision-making process from experience-driven to a computable constrained optimization problem.

[0062] This invention integrates three risk categories—distance, collision time, and relative velocity—into a single, calculable risk index, enabling quantitative safety assessment of candidate control actions. Furthermore, the risk constraint unit incorporates adaptive parameter adjustment rules, dynamically adjusting weight parameters based on the environmental complexity of the target scenario. , , For example, in high-speed scenarios Take 0.5, Take 0.4, Set to 0.1; in urban road scenarios Take 0.6, Take 0.3, Take 0.1; in rain / snow / fog weather, distance to risk Increase by 0.2, collision time risk Increase by 0.1; when the agent travels at night, the distance risk... Increase by 0.15 to balance distance safety and collision time safety.

[0063] Furthermore, the security interception mechanism is an overmatch rejection mechanism.

[0064] The security interception unit is specifically configured to perform: When the risk score is less than or equal to a preset threshold, a candidate control action is executed; When the risk score exceeds a preset threshold, the overmatch rejection mechanism is triggered, the safety distance constraint is actively expanded and the decision problem is reconstructed into a constraint optimization problem, generating the safest replacement action with the lowest risk.

[0065] Furthermore, when proactively expanding the safety distance constraint and reconstructing the decision problem into a constrained optimization problem to generate the safest alternative action with the least risk, the safety interception unit is specifically configured to execute: The safety distance constraint is dynamically expanded based on the risk score using a second calculation formula; the second calculation formula is as follows: ; in, This indicates the expanded safety distance. Indicates the original safe distance.

[0066] This represents the safety expansion coefficient, which is dynamically adjusted based on the risk score. =1+0.5 ( θ), λ∈[1.2,2.0]; express Time-based candidate control actions The corresponding risk score, where θ represents a preset threshold. The higher the risk score, the lower the threshold. The larger θ is, the larger λ is, and the greater the extended safety distance range.

[0067] The decision-making process is restructured into a minimum risk optimization problem with extended safety constraints based on the third calculation formula. The third calculation formula is as follows: ; in, This represents the optimal safe replacement action obtained from the solution. This means finding the action with the least risk within the available action space. This indicates compliance with a set of safety constraints. This represents the expanded set of safety constraints.

[0068] Furthermore, the security interception unit is also pre-configured with a security backup strategy. When the reconstruction decision problem becomes a constraint optimization problem and no safe replacement action can be obtained, the security interception unit is configured to trigger the security backup strategy.

[0069] In other words, when the constraint optimization solution cannot obtain the optimal safe replacement action, the safety backup strategy is triggered to forcibly replace the original candidate control action that has high risk, so as to achieve a safety backup in high-risk scenarios.

[0070] Furthermore, the experience evolution unit is specifically configured to execute: After a candidate decision is executed, the decision-making process is encapsulated as an experience tuple and stored in a distributed experience repository. This distributed experience repository includes multiple historical experience tuples encapsulated from historical decision-making processes.

[0071] The skill library includes basic skills and composite skills. Basic skills are predefined by developers and have pre-defined, fixed strategy mapping functions whose parameters are not updated with experience. Composite skills are generated by combining basic skills and stored in a structured quintuple format; their parameters can be optimized through iterative experience.

[0072] Specifically, after the candidate decision is executed, the empirical evolution unit encapsulates each decision process into an empirical tuple. Specifically, it is expressed as follows: ; in, , Indicates the outcome of the decision's execution. This indicates that the current state of selecting this control action has successfully achieved the expected task effect, and can serve as a reusable and excellent experience for similar scenarios in the future; This indicates that selecting this control action in the current state cannot achieve the expected task requirements, which is a negative failure experience, and should be avoided in similar scenarios in the future.

[0073] This indicates the response time from the issuance of a decision to its completion.

[0074] The encapsulated experience tuples will be continuously added to the distributed experience repository D. Specifically, .

[0075] For a new scene state in the current global structured state vector of the agent, the most matching historical experience is obtained from the distributed experience base through similarity retrieval.

[0076] Specifically, when the intelligent agent operation and maintenance system faces new scenario states At that time, the most matching historical experience is obtained through similarity retrieval within the distributed experience base: ; in, This represents the most matching historical experience retrieved. This represents the state similarity measurement function. This represents the historical state recorded in the historical experience tuple e.

[0077] After determining the best-matching historical experience, the Frenet coordinate system trajectory translation and scaling algorithm is used to perform scene adaptive correction between the new scene state and the best-matching historical experience, so as to determine the candidate control action corresponding to the new scene state based on the skill executed by the best-matching historical experience.

[0078] The specific steps for scene adaptive correction between the new scene state and the best-matching historical experience include: S1. Map the sequence of trajectory points in the global coordinate system from the determined best-matching historical experience to the Frenet coordinate system, and decompose each trajectory point to obtain the vertical coordinate s and horizontal coordinate d, thereby realizing the transformation from the global trajectory to the Frenet coordinate system.

[0079] S2. Based on the differences between the new scenario state and the historical scenario corresponding to historical experience, the longitudinal and lateral lengths of the trajectory are scaled and corrected to obtain the scaled Frenet trajectory.

[0080] Vertical trajectory scaling correction is achieved through a vertical length scaling formula. ; in, It represents the longitudinal length of a trajectory in historical experience. This represents the length of the vertical trajectory after adapting to the new scene state. This represents the baseline speed of the agent in the historical scenario corresponding to historical experience. This represents the actual speed of the agent in the new scenario state.

[0081] Lateral trajectory scaling correction is achieved through a lateral length scaling formula. ; in, This represents the lateral offset of the trajectory in historical experience. This represents the new lateral offset after adapting to the new scene state. This represents the width that restricts the agent's lateral movement within the historical context corresponding to historical experience. This represents the actual width that restricts the agent's lateral movement in the new scene state.

[0082] S3, scale the Frenet trajectory ( , Transform back to the global coordinate system to generate new candidate control actions.

[0083] S4. Perform a security assessment on the newly generated candidate control actions. If the risk score of the security assessment exceeds the preset threshold, trigger the security interception mechanism.

[0084] In other words, when encountering a new scenario state, the system can search for the historical state that is closest to the new scenario state in the historical experience library and reuse the complete experience of that historical state.

[0085] Furthermore, the experience evolution unit is also configured to perform the following: periodically extract high-value experience tuples from the distributed experience base, generate new composite skills using clustering algorithms, and add the composite skills that have been verified through automated simulation testing to the skill base, thereby updating the skills in the skill base.

[0086] High-value experience tuples refer to experience tuples with a success rate greater than the first preset value and a complexity greater than the second preset value.

[0087] In this embodiment, the first preset value is 95%, and the second preset value is 0.7.

[0088] Specifically, the operations and maintenance management team extracts the top 1000 high-value experience tuples (success rate > 95% and complexity > 0.7) from the experience base each week, generates new composite skills through clustering algorithms, and adds the new composite skills to the skill base after they are verified by automated simulation testing. This process only updates the structured quintuples in the skill base and does not involve any modification to the basic model parameters.

[0089] The intelligent agent operation and maintenance system based on the self-evolution framework of this invention adopts a modular AgentOps architecture with a magnetic layer. It is the first to use the runtime support layer as the core execution and security assurance layer, realizing the integrated design of dynamic skill assembly and security introspection, and solving the problem of rigidity in traditional architecture.

[0090] The intelligent agent operation and maintenance system based on the self-evolution framework of the present invention supports a dynamic skill assembly mechanism based on structured quintuples. Skills are abstracted into standardized quintuples, which support cross-scenario on-demand combination and improve generalization ability.

[0091] The intelligent agent operation and maintenance system based on the self-evolutionary framework of the present invention supports a security introspection mechanism and an over-match rejection mechanism, constructs a computable risk constraint system, actively expands security constraints and generates alternative decisions in high-risk scenarios, and achieves a balance between security and performance.

[0092] The intelligent agent operation and maintenance system based on the self-evolutionary framework of the present invention supports the self-evolutionary mechanism of nonparametric experience transfer and skill base distillation separation. The basic model parameters are fixed, and the skill base is optimized only through iterative optimization of structured experience tuples, thereby avoiding parameter drift from the root and reducing operation and maintenance costs.

[0093] The intelligent agent operation and maintenance system based on the self-evolutionary framework of this invention supports the full-process automated closed loop of DevSecOps, integrates CI / CD, automated simulation testing, OTA updates, quality access control and version rollback, and realizes unmanned operation and maintenance of intelligent agents throughout their entire life cycle; its overall decision-making closed loop is "state modeling - skill combination - risk constraint - over-matching interception - non-parametric experience evolution", which can form a self-evolutionary closed loop in parallel with short-term decision optimization and long-term capability evolution.

[0094] The intelligent agent operation and maintenance system based on the self-evolutionary framework provided in this invention is applicable to the full life cycle management, capability evolution and security operation and maintenance of intelligent agents in various complex dynamic environments. The system has high versatility and can be widely applied to complex intelligent agent scenarios such as autonomous driving and software development.

[0095] Secondly, embodiments of the present invention also provide an application method for an intelligent agent operation and maintenance system based on a self-evolutionary framework. This method is based on the four-layer modular hierarchical architecture of the intelligent agent operation and maintenance system based on the self-evolutionary framework described above, and constructs a closed-loop decision-making mechanism to realize safe decision-making for autonomous driving and autonomous evolution of intelligent agents.

[0096] In other words, when the intelligent agent operation and maintenance system based on the self-evolutionary framework of the present invention is applied to autonomous driving decision control, it constructs a self-evolutionary closed loop driven by a safety introspection mechanism based on the four-layer modular hierarchical architecture of the intelligent agent operation and maintenance system, integrates short-term decision optimization and long-term capability evolution, and realizes safe decision-making and autonomous evolution of intelligent agents in autonomous driving.

[0097] Specifically, it includes: When performing target scene perception and dynamic skill combination, the driving agent in the application layer uses a multimodal sensor suite (camera, LiDAR, millimeter-wave radar) to achieve high-precision perception of the current driving scene, extracting environmental information such as lane lines, traffic signs, the motion status of surrounding vehicles, and road topology. The dynamic assembly module in the runtime support layer dynamically selects and combines a suitable set of driving skills from the skill library based on the scene environment perception results and the target driving task (cruising / obstacle avoidance / lane changing, etc.), and sends it to the driving agent as the basis for decision-making.

[0098] When making safety decisions in an Actor-Critic dual-agent system, such as Figure 3 As shown, the Actor-Critic dual-agent safety decision-making mechanism adopted in this invention is specifically as follows: Based on an assembled skill set and scene perception information, the Actor plans and generates driving trajectories and control commands (steering, accelerator, or braking) to ensure the smoothness of driving tasks and basic compliance with traffic rules.

[0099] The Critic agent performs a full-dimensional safety review (security assessment) on the candidate decision instructions (candidate control actions) generated by the Actor driving agent.

[0100] For example: using a risk prediction network to calculate the real-time risk score of a decision instruction, and combining this with a vehicle dynamics physical model to perform worst-case analysis, if the risk score value of the candidate decision instruction... If the risk score of the candidate decision instruction exceeds a preset threshold θ, an overmatch rejection mechanism is immediately triggered, replacing the candidate decision instruction with a safe replacement action. If the value is less than or equal to the preset threshold θ, the candidate decision instruction will be allowed to be sent to the vehicle's underlying control system; unsafe decisions that are blocked will also be recorded in real time and fed back to the status manager.

[0101] The specific formula for calculating the risk score of the risk prediction network is as follows: ; in, The risk score is indicated by the number of vehicles around the vehicle participating in the risk assessment. The higher the risk score, the more dangerous the corresponding driving decision instruction. m represents the total number of vehicles around the vehicle participating in the risk assessment, and i represents the current sequence number of the vehicles around the vehicle participating in the risk assessment.

[0102] The safety distance penalty factor is determined by the relative speed between the vehicle and the i-th surrounding intelligent agent. System response time Perception error The calculations yielded the following specific results: .

[0103] , Let represent the normalized lateral distance and longitudinal distance between the vehicle and the i-th surrounding vehicle, respectively.

[0104] When making short-term decision-making optimizations in real time, the driving agent executes the decision instructions after the release, while recording key data such as control errors, scene changes, and passenger comfort feedback in real time. Through the driving playback mechanism, without updating the model parameters, the sensitivity of the perception model, the preference parameters of the planning algorithm, and the compensation strategy in the execution stage are gradually optimized to achieve implicit adaptation to the current driving scenario and generate more human-like and more efficient driving behavior.

[0105] During long-term capability auto-evolution, the state manager stores driving logs, decision-making strategies, and result feedback from successfully handled complex scenarios in the skill library as structured experience tuples. The operations and maintenance management team periodically extracts typical scenarios (including successfully handled scenarios and intercepted unsafe scenarios) from the skill library and state manager. After verification through a simulation testing platform, the model training and distillation modules iteratively optimize the skill library to form new driving skills. These new skills are remotely deployed via the CI / CD pipeline and OTA update module, with the version control module monitoring the entire process. If the performance of a new skill fails to meet the quality threshold after deployment, an automatic rollback is immediately triggered to ensure system stability. When encountering a new scenario state, the system does not directly modify model parameters. Instead, it retrieves the most similar successful historical cases from the skill library and generates a decision response through scenario similarity matching and strategy migration, avoiding unknown risks introduced by model parameter drift.

[0106] Throughout the decision-making and evolution process of the intelligent agent operation and maintenance system based on the self-evolution framework, the infrastructure layer realizes real-time data collection and storage, the runtime support layer continuously feeds back status data and abnormal events to the operation and maintenance management layer, and the operation and maintenance management layer completes the iteration, testing, deployment and version management of skills / models, forming a closed-loop data process of perception-decision-execution-feedback-optimization, realizing unmanned and automated operation and maintenance of autonomous driving intelligent agents.

[0107] The intelligent agent operation and maintenance system based on the self-evolution framework of this invention is applied to realize the safety decision-making and autonomous evolution of intelligent agents in autonomous driving. Through a four-layer modular layered architecture and dynamic assembly engine, it realizes the on-demand capability assembly of autonomous driving intelligent agents. The skill library has the ability to generalize across traffic culture, cross-regional regulations, and cross driving scenarios. The system can dynamically adjust the skill combination according to the actual road conditions and task requirements, which greatly improves the adaptability to complex and unknown scenarios.

[0108] This invention constructs a self-evolving closed loop that combines short-term decision optimization with long-term capability evolution. Through structured experience tuple storage and skill library iteration, it realizes dynamic expansion and optimization of driving skills. It can adapt to new scenario states without retraining the model, thus solving the problem of insufficient processing capability of traditional systems for long-tail edge scenarios.

[0109] This invention introduces the DevSecOps concept into the management of autonomous driving agents. Through CI / CD pipelines, automated simulation testing, OTA updates, version control, and quality gates, it enables continuous integration, deployment, testing, and rollback of skills / models, replacing manual management of the entire lifecycle of agents and significantly reducing the human and time costs of development, deployment, and maintenance.

[0110] This invention designs a safety introspection mechanism based on the Actor-Critic dual-agent system, which combines a risk prediction network and an overmatch rejection mechanism to intercept unsafe decisions in real time. At the same time, it maintains driving efficiency while ensuring zero collisions by making "proactive safety" tactical choices (such as slightly crossing the line in extreme scenarios to avoid collisions). This solves the problem of safety and performance imbalance caused by single-dimensional evolution.

[0111] This invention generates decisions through scene similarity matching and strategy transfer during long-term evolution, rather than directly modifying the neural network weights. This ensures that each decision follows the forward inference logic of the original network, fundamentally avoiding the unknown security risks introduced by model parameter drift and improving the stability and reliability of the system operation.

[0112] The four-layer AgentOps architecture of this invention adopts component decoupling and standardized interface design, which can flexibly expand the skill library and intelligent agent type according to different autonomous driving models and application scenarios. At the same time, it can be extended to other complex intelligent agent application fields such as intelligent robots and industrial intelligent decision-making, and has broad engineering application value.

[0113] Example 1 Embodiment 1 of this invention verifies the system's performance in a high-speed scenario using the CARLA simulator. For example, the effectiveness of the intelligent agent operation and maintenance system based on this invention in making autonomous driving decisions is verified in the Town04 four-lane highway scenario of the CARLA 0.9.15 simulator.

[0114] Specifically, the simulation environment is set as a high-density random traffic environment, with NPC vehicles having a 20% tendency to speed and a 20% probability of random lane changes. Three typical test scenarios are designed: S1 random high-density traffic, S2 roadblocks, and S3 aggressive traffic flow. The performance indicators (including average speed, collision rate, violation rate, and safety score) of the autonomous driving decision-making based on the intelligent agent operation and maintenance system of this invention are compared with the baseline of CARLA native traffic manager (rule-driven).

[0115] The intelligent agent operation and maintenance system based on the self-evolutionary framework of this invention is deployed on a workstation equipped with an NVIDIA RTX 4090 GPU. The inference and training framework is built using PyTorch 2.2.0 and CUDA 12.4. Real-time data interaction between the simulator and the system is achieved through ROS2. The skill library is pre-configured with basic and composite driving skills such as cruise, lane changing, obstacle avoidance, and emergency avoidance. The safety self-reflection module has a preset risk score threshold θ=0.8, and the quality access control module has preset performance thresholds of 0% collision rate and ≥90 safety score.

[0116] Simulation results show that in the S1 random high-density traffic scenario, the simulation results of autonomous driving decision-making using the intelligent agent-based operation and maintenance system of this invention are: average speed 25.25 km / h, collision rate 0%, violation rate 10%, and safety score 99. Compared with the simulation results of the traditional baseline method, which have an average speed of 29.30 km / h, a collision rate of 20%, and a safety score of 89, this invention maintains a safe following distance through precise longitudinal speed control, with no collisions occurring and a few violations being tactical lane boundary fine-tuning, which is far superior to the high collision rate of the traditional baseline method.

[0117] In the S2 obstacle scenario, the simulation results obtained by implementing autonomous driving decision-making using the present invention are: average speed 15.74 km / h, collision rate 0%, violation rate 0%, and safety score 100. Compared with the simulation results of the traditional baseline method, which have an average speed of 4.96 km / h, a collision rate of 0%, and a safety score of 100, the present invention uses a dynamic assembly engine to call obstacle avoidance skills and initiates active lane changing based on semantic cues. This avoids the "frozen robot" problem of the baseline method, maintains higher driving efficiency under the premise of zero collision, and ensures smooth traffic flow.

[0118] In the S3 aggressive traffic flow scenario, the simulation results obtained by using the present invention to realize autonomous driving decision-making are: average speed 42.38 km / h, collision rate 0%, violation rate 20%, and safety score 98.2 points. Compared with the simulation results of the traditional baseline method, which have an average speed of 31.73 km / h, a collision rate of 30%, and a safety score of 84 points, the safety introspection module of the present invention can quickly infer the sudden cutting intention of surrounding vehicles, intercept dangerous decisions through the over-match rejection mechanism, and make a small number of violations an active safety tactical choice in extreme scenarios, thus achieving the dual goals of zero collision and high-speed driving. In contrast, the baseline method has a collision rate as high as 30% due to the lack of intention reasoning and active obstacle avoidance capabilities.

[0119] After 10 rounds of simulation experiments, the intelligent agent operation and maintenance system stores the successfully handled complex scenarios such as aggressive entry and high-speed obstacle avoidance into the skill library as structured experience tuples through self-evolution closed loop. The iterated skill library reduces the decision response time by 30% and improves the smoothness of control commands by 40% in the same scenario, significantly improving passenger comfort. Moreover, no human intervention is required throughout the process, realizing automated capability evolution.

[0120] Example 2 Embodiment 2 of this invention verifies a mixed urban expressway-highway scenario based on the CARLA simulator. For example, the intelligent agent operation and maintenance system based on the self-evolutionary framework of this invention is deployed on a Level 4 autonomous passenger vehicle and applied to a mixed urban expressway and highway scenario.

[0121] Specifically, the computing unit in the infrastructure layer is the vehicle-mounted computing unit, which integrates sensors such as autonomous driving domain controllers, LiDAR, cameras, and millimeter-wave radar; a private cloud is deployed on the cloud server to achieve real-time data interaction between the vehicle and the cloud via 5G; and a data storage memory pool enables real-time storage of local driving logs and synchronous sharing of knowledge in the cloud.

[0122] Based on the traffic rules of the target driving area (such as a speed limit of 120km / h on highways and 80km / h on urban expressways), initialize the skills in the skill library, such as cruise, lane changing, obstacle avoidance, and following, and set the risk score threshold and quality access control indicators for the safety self-reflection module.

[0123] During actual road operation, the dynamic assembly engine automatically assembles a skill set based on real-time scenarios (such as high-speed cruising, deceleration of the vehicle in front, and sudden road obstacles). The safety self-reflection module monitors decision-making instructions throughout the process and intercepts unsafe operations. Successfully handled complex scenarios (such as high-speed overtaking, following another vehicle in the rain, and avoiding obstacles in construction sections) are recorded as structured experience tuples and uploaded to the cloud skill library. After verification through simulation testing, the cloud operation and maintenance management team iteratively optimizes the skill library and pushes it to the vehicle terminal via OTA, realizing the continuous evolution of the vehicle's driving capabilities.

[0124] After 30 rounds of real-vehicle verification, the system achieved zero collisions and zero major violations. In typical scenarios such as sudden vehicle incursions and avoidance of construction zones, the intelligent agent operation and maintenance system's response time was ≤0.5s, significantly improving decision-making smoothness and human-like driving performance. The system automates the entire process of skill iteration, simulation testing, and version deployment, reducing operation and maintenance costs by 60% compared to traditional solutions. Therefore, the intelligent agent operation and maintenance system of this invention, used for autonomous driving safety decision-making, demonstrates safety, stability, and engineering practicality in real-vehicle scenarios.

[0125] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of the present invention is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0126] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions within the technical scope disclosed in the present invention should be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An intelligent agent operation and maintenance system based on a self-evolutionary framework, characterized in that, The system adopts a four-layer modular architecture, consisting of an intelligent agent application layer, a runtime support layer, an operation and maintenance management layer, and an infrastructure layer, from top to bottom. Each layer realizes data interaction, command flow, and status feedback through standardized APIs and event-driven mechanisms, thereby achieving a closed-loop decision-making mechanism for unmanned operation and maintenance and autonomous evolution of the intelligent agent throughout its entire lifecycle. The agent application layer is used to deploy target agent instances that directly interact with the target scenario; the runtime support layer includes a skill library, a dynamic assembly engine, a security introspection module, a policy service module, and a state manager, which are used to provide a standardized operating environment and core functional services for the target agents deployed in the agent application layer. The core functional services include encapsulating underlying resource differences, dynamically assembling skills, and real-time interception of insecure decisions; the operation and maintenance management layer is used to realize the automated management of the entire lifecycle of the target agent. The infrastructure layer is used to provide physical and virtual resource support for the operation and maintenance system; The construction of the closed-loop decision-making mechanism includes: The state modeling unit is used to perform unified modeling of the agent's operating scenario based on the target agent and the target scenario, obtain the agent's current global structured state vector, and retrieve a set of candidate skills in the skill base based on the agent's current global structured state vector through a similarity function; The skill combination unit is used to abstract each agent skill of the target agent into a structured quintuple, and to combine candidate skills in the candidate skill set to generate candidate control actions; the structured quintuple of the agent skill includes skill application conditions, skill execution pre-constraints, skill control parameter space, policy mapping function, and skill execution effect evaluation index; The risk constraint unit is used to perform a safety assessment of the candidate control actions based on a decision constraint mechanism using a risk function. The security interception unit is used to trigger the security interception mechanism when the risk score value of the security assessment exceeds a preset threshold, dynamically expand the security constraints and generate the least risky security replacement action; An experience evolution unit is used to, when the agent's current global structured state vector is in a new scenario state, adopt a non-parametric experience transfer mechanism based on the executed candidate decisions, and update the skills in the skill library that best match the candidate decisions of the new scenario state to obtain the candidate control actions corresponding to the new scenario state; the candidate decisions are either the candidate control actions or the safe replacement actions. The state modeling unit is specifically configured to execute: A unified model of the agent's operating scenario is performed based on the target agent and the target scenario. The multimodal perception data of the agent's operating scenario is mapped into the agent's current global structured state vector to describe the current environment and task constraints. The agent's current global structured state vector includes the agent's own position state, the agent's own velocity state, the position state of the entities surrounding the agent, the velocity state of the entities surrounding the agent, the environmental topology, the set of rule constraints, and the set of objects perceived by the agent. Using the agent's current global structured state vector as input, a weighted cosine similarity function is used to compare the agent's historical scene states, and the most similar target historical scene states are selected through a Top-K filtering operation. These are then mapped to the skill library to obtain a set of candidate skills corresponding to the target historical scene states.

2. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 1, characterized in that, The experience evolution unit is specifically configured to execute: After the candidate decision is executed, the current decision-making process is encapsulated as an experience tuple and stored in a distributed experience repository; the distributed experience repository includes multiple historical experience tuples encapsulated from historical decision-making processes. For a new scene state in the current global structured state vector of the agent, the most matching historical experience is obtained from the distributed experience base through similarity retrieval; The Frenet coordinate system trajectory translation and scaling algorithm is used to perform scene adaptive correction between the new scene state and the best matching historical experience, so as to determine the candidate control action corresponding to the new scene state based on the skill executed according to the best matching historical experience.

3. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 2, characterized in that, The experience evolution unit is also configured to execute: High-value experience tuples are periodically extracted from the distributed experience base, new composite skills are generated using clustering algorithms, and composite skills verified through automated simulation testing are added to the skill base to update the skills in the skill base; the high-value experience tuples refer to experience tuples with a success rate greater than a first preset value and a complexity greater than a second preset value.

4. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 1, characterized in that, The skill combination unit includes components specifically configured to perform: Each agent skill of the target agent is abstracted into a structured quintuple; Based on the first calculation formula, the candidate control action corresponding to the weighted fusion of all candidate skills in the candidate skill set is determined. The first calculation formula is as follows: ； in, Indicates candidate control actions; The current control cycle represents the timing of the agent's control. This represents the total number of candidate skills in the candidate skill set. Indicates the current candidate skill number; This represents the weight coefficient of the current candidate skill. Indicates the first The policy mapping function for each candidate skill. express The current global structured state vector of the agent at time step.

5. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 4, characterized in that, The skill combination unit also has a pre-set multi-skill conflict resolution mechanism; When there is a conflict in the control instructions of multiple candidate skills in the candidate skill set, the skill combination unit triggers a multi-skill conflict resolution mechanism and generates candidate control actions by using a priority sorting and constraint fusion method. The priority of the candidate skills in the candidate skill set, from highest to lowest, is as follows: safety skills, task skills, and comfort skills.

6. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 1, characterized in that, The risk constraint unit is specifically configured to execute: A decision constraint mechanism based on a risk function is used to assess the safety of the candidate control actions and obtain a risk score; the risk function is specifically: ； in, express Time-based candidate control actions The corresponding risk score, This represents the total number of target entities present within the current sensing range. Indicates the target agent and the first The relative distance between target entities This represents the remaining time before the target agent collides with the i-th target entity. This represents the speed of the target intelligent agent. This indicates the speed of the target entity. It is a very small positive number. , , These are the weighting parameters for distance risk, collision time risk, and relative speed risk, respectively.

7. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 6, characterized in that, The security interception mechanism is an over-match rejection mechanism; the security interception unit is specifically configured to execute: When the risk score is less than or equal to a preset threshold, the candidate control action is executed; When the risk score is greater than the preset threshold, the overmatch rejection mechanism is triggered, the safety distance constraint is actively expanded and the decision problem is reconstructed into a constraint optimization problem, and the safe replacement action with the lowest risk is generated.

8. The intelligent agent operation and maintenance system based on a self-evolutionary framework according to claim 7, characterized in that, When proactively expanding the safety distance constraint and reconstructing the decision problem into a constraint optimization problem to generate the safest alternative action with the lowest risk, the safety interception unit is specifically configured to execute: The safety distance constraint is dynamically expanded based on the risk score using a second calculation formula; the second calculation formula is specifically as follows: ； in, This indicates the expanded safety distance. Indicates the original safe distance; This represents the safety expansion coefficient, which is dynamically adjusted based on the risk score value. =1+0.5 ( θ), λ∈[1.2,2.0]; express Time-based candidate control actions The corresponding risk score, where θ represents the preset threshold; The decision-making process is restructured into a minimum risk optimization problem with extended safety constraints based on the third calculation formula, which is specifically: ； in, This represents the optimal safe replacement action obtained from the solution. This means finding the action with the least risk within the available action space. This indicates compliance with a set of safety constraints. This represents the expanded set of safety constraints.

9. An application method for an intelligent agent operation and maintenance system based on a self-evolutionary framework, characterized in that, The method is based on the four-layer modular hierarchical architecture of the intelligent agent operation and maintenance system based on the self-evolution framework as described in any one of claims 1 to 8, and constructs a closed-loop decision-making mechanism to realize safe decision-making for autonomous driving and autonomous evolution of intelligent agents.

Images