An autonomous intelligent agent-oriented candidate action safety release and closed-loop update method and system, and a storage medium

Abstract

The application discloses an autonomous intelligent agent-oriented candidate action safety release and closed-loop updating method and system and a storage medium, and belongs to the technical field of intelligent control and intelligent agent execution management. The method acquires task context data, current state data and risk constraint data, and constructs a current feature vector. According to the similarity with a historical feature library, a candidate action instruction is generated among historical action direct reuse, local optimization and re-optimization solution. The target state after execution is predicted in a boundary checking sandbox, and the minimum distance of the target state to a safety boundary is calculated. When the minimum distance does not meet the safety margin requirement, a correction action instruction is generated according to the action variable sensitivity and checked again. When the checking again still does not meet the release condition, a conservative action instruction is generated. The actual execution result, the state after execution, the candidate action generation path, the boundary checking result and the final release conclusion are written back to the historical feature library and the historical decision cache, so as to update the credibility score, the reuse priority, the cache level and the path selection threshold parameter, and the updated result is used for subsequent candidate action generation path judgment and action release decision. The scheme is helpful for reducing the dependence on artificial review, improving the automatic release efficiency and closed-loop learning ability.

Description

Technical Field

[0001] This application relates to the fields of intelligent control and intelligent agent execution governance technology, and in particular to a method, system and computer-readable storage medium for the safe release and closed-loop update of candidate actions for autonomous intelligent agents. Background Technology

[0002] With the development of large models, intelligent agent orchestration frameworks, rule engines, and real-time perception technologies, autonomous intelligent agents that can autonomously generate action suggestions and drive target objects to perform actions based on task objectives, task context, and environmental states are being gradually applied to scenarios such as industrial operation and maintenance, building control, park scheduling, energy management, security linkage, and enterprise process automation.

[0003] In the above scenario, the value of autonomous agents lies not only in their ability to generate action suggestions, but also in their ability to release actions in a short period of time. Action release means that before a candidate action generated by an autonomous agent enters the business system interface, control interface, or execution interface, it has undergone action rationality judgment, boundary risk verification, and necessary action correction, so that the candidate action can be executed automatically without relying on manual review each time.

[0004] In existing technologies, one type of solution typically outputs action instructions directly through rule bases, template libraries, or historical case retrieval. While this type of solution offers a fast response time, in scenarios with insufficient historical sample coverage, rapidly changing operating environments, or highly sensitive action consequences, directly reusing historical actions can easily lead to a mismatch between the action and the current state.

[0005] Another approach typically involves re-performing optimization solutions or complex simulations to verify the actions output by the autonomous agent. While this approach can reduce the probability of issuing erroneous actions to some extent, in online execution scenarios, full solutions or complex simulations often lead to high latency, making it difficult to meet the autonomous agent's requirement for rapid action release.

[0006] Another type of solution can record the effects before and after the execution of candidate actions, but it is more limited to log retention or post-event auditing. It lacks a continuous learning mechanism that unifies the action release result, actual execution result and post-execution status back to the historical knowledge system and directly affects the generation of the next round of actions.

[0007] Therefore, the existing technology has at least the following shortcomings: First, it lacks a hierarchical generation mechanism for candidate actions for autonomous agents, making it difficult to balance the efficiency of historical experience reuse with the ability to solve unfamiliar scenarios; Second, it lacks an action release mechanism based on safety boundary distance rather than a simple pass / reject, resulting in a lack of calculable fine-tuning basis for candidate actions when they approach the risk boundary; Third, it lacks a closed-loop update mechanism that continuously writes the action release result and the post-execution result back to the historical knowledge system, making it difficult for the autonomous agent to continuously correct its subsequent action release strategy.

[0008] Therefore, there is an urgent need for a method, system, and storage medium that can simultaneously take into account the efficiency of action generation, the security of action release, and the ability to continuously learn after execution, as well as the ability to securely release and update candidate actions in a closed loop. Summary of the Invention

[0009] This application provides a method, system, and storage medium for the safe release and closed-loop update of candidate actions for autonomous intelligent agents. By constructing a collaborative link of "similarity hierarchical generation of candidate actions, boundary distance-driven safe release and fine-tuning, and execution result writing back to drive subsequent path selection updates", it solves the technical problems of insufficient reliability of action release, high dependence on manual review, and incomplete learning closed loop in existing autonomous intelligent agents in high-risk business scenarios.

[0010] In a first aspect, this application provides a method for the safe release and closed-loop update of candidate actions for autonomous agents, comprising: acquiring task context data, target system current state data, and risk constraint data generated by the autonomous agent, and constructing a current feature vector; calculating the similarity between the current feature vector and multiple historical features in a historical feature library to obtain a target similarity and determine the target historical features; and determining a candidate action instruction generation path from a direct reuse path of historical actions, a local optimization path with historical actions as the initial solution, and a re-optimization solution path based on the relationship between the target similarity and a first preset threshold and a second preset threshold.

[0011] The candidate action command is input into the boundary verification sandbox. Based on the current set of valid constraints, the target prediction state after executing the candidate action command is predicted, and the minimum distance from the target prediction state to the safety boundary is calculated. Based on the relationship between the minimum distance and the safety margin, it is determined whether to release directly, release after fine-tuning the action, or release the conservative action. When the minimum distance is less than the safety margin, a corrected action command is generated based on the difference between the safety margin and the minimum distance, the sensitivity of the action variable, the constraint coupling relationship, and the preset adjustment upper limit, and then re-verified.

[0012] The system obtains the actual execution result and post-execution status corresponding to the issued action command, and writes the actual execution result, the post-execution status, the target similarity, the candidate action command generation path, the boundary verification result, and the final release conclusion back to at least one of the historical feature library and historical decision cache to update the credibility score, reuse priority, cache level, and path selection threshold parameters corresponding to the hit historical features. When generating subsequent candidate actions, the system re-executes the similarity layered path determination and action release decision based on the updated credibility score, reuse priority, cache level, and path selection threshold parameters.

[0013] In one implementation, the task context data includes at least one of the following: target task identifier, business object identifier to be executed, action parameters generated by the autonomous agent, action description generated by the autonomous agent, action confidence generated by the autonomous agent, action source agent identifier, and upstream triggering event identifier.

[0014] In one implementation, the step of calculating the similarity between the current feature vector and historical features in the historical feature library includes: calculating the similarity score between the current feature vector and multiple historical features; sorting them from high to low according to the similarity score; selecting the historical features in the top K positions of the sorting result as candidate historical features; and selecting the historical feature with the highest similarity score among the candidate historical features as the target historical feature.

[0015] In one implementation, when the target similarity is greater than or equal to a first preset threshold, historical action instructions corresponding to the historical features of the target are directly extracted from the historical decision cache; when the target similarity is less than the first preset threshold but greater than or equal to a second preset threshold, historical action instructions corresponding to the historical features of the target are extracted, and local optimization is performed using the historical action instructions as the initial solution; when the target similarity is less than the second preset threshold, re-optimization is performed based on the current state data.

[0016] In one implementation, the boundary verification sandbox is pre-set with at least one of physical boundary constraints, security boundary constraints, and business rule boundary constraints.

[0017] In one implementation, when the minimum distance is less than the safety margin, a corrective action command is generated based on the difference between the minimum distance and the safety margin, the sensitivity of the action variable, the constraint coupling relationship, and the preset adjustment upper limit, and the corrective action command is re-verified.

[0018] In one implementation, the write-back update includes updating at least one of the following: the credibility tag of the hit historical feature, the reuse priority, the cache level, the threshold parameter, the action generation path marker, or the action release log; the updated threshold parameter is at least partially involved in the determination of the subsequent candidate action generation path.

[0019] Secondly, this application provides a candidate action safe release and closed-loop update system for autonomous intelligent agents, including: a feature construction module, a similarity retrieval module, a candidate action generation module, a boundary verification module, an action release module, and a closed-loop update module; wherein, the feature construction module is used to acquire task context data, current state data, and risk constraint data and construct a current feature vector.

[0020] Thirdly, this application provides a computer-readable storage medium having program instructions stored thereon, which, when executed by a processor, implement the above-described method.

[0021] In some alternative implementations, the above method can be performed by a computing device including a memory and a processor, wherein the memory is used to store program instructions and the processor is used to execute the program instructions to implement the above method.

[0022] In some alternative implementations, the above method can be implemented in the form of a computer program product, which includes program code that, when run on a computing device, causes the computing device to execute the above method.

[0023] Compared with existing technologies, this application has at least the following technical effects: First, by generating candidate actions through similarity layering, it can take into account both the need for rapid release in repetitive scenarios and the need for re-solving in unfamiliar scenarios, reducing online solution latency and the frequency of manual review; Second, by using an action release and fine-tuning mechanism based on security boundary distance, the candidate action can be extended from a simple pass / rejection judgment to a computable security correction process, thereby improving the success rate of automatic release; Third, by writing the action release results and post-execution results back to the historical knowledge system, and directly influencing the subsequent candidate action generation and release decisions with the updated credibility score, cache level, and path selection threshold, a continuous learning closed loop suitable for high-risk business scenarios can be formed.

[0024] The terms used in this application are defined as follows.

[0025] An autonomous intelligent agent is an intelligent processing entity that can autonomously generate action suggestions based on task objectives, task context, and environmental conditions, and interact with business or physical systems through external interfaces. The autonomous intelligent agent can be a single agent or an agent node in a multi-agent collaborative system.

[0026] Task context data refers to a set of data related to the task currently to be executed by the autonomous agent, including at least one of the following: task identifier, task type, target object identifier, action target, action parameters, action reason, action confidence, action source, upstream event information, or scheduling source information.

[0027] Current state data refers to the data set that characterizes the operating state of the target system at the time of candidate action generation, including at least one of the following: device operating state, environmental parameters, business object state, resource occupancy state, permission state, risk marker, or dependency relationship between objects.

[0028] A historical feature database is a knowledge storage structure used to store historical task features, historical state features, historical action instructions, boundary verification results, and corresponding execution results. Historical feature databases can be implemented using searchable data structures such as databases, vector index libraries, and graph index libraries.

[0029] Historical decision cache refers to a data structure used to cache historical action instructions with high reusability. In some implementations, historical decision cache can be implemented in a hierarchical caching manner to improve the generation speed of candidate actions in highly similar scenarios.

[0030] A boundary verification sandbox is a verification environment used to predict the consequences of an action and determine whether it meets the release conditions based on preset boundary constraints before the action is actually issued. The boundary verification sandbox can call simplified state transition models, rule engines, lightweight simulation models, or digital twin models. In some implementations, the boundary verification sandbox focuses on verifying state variables and boundary margins directly related to action release, rather than performing a full high-fidelity simulation of the target system.

[0031] A security boundary refers to the set of constraint boundaries that a target system must not breach during the execution of its actions. The security boundary can consist of at least one of physical boundary constraints, security boundary constraints, and business rule boundary constraints. In some implementations, risk constraint data is a set of data used to characterize the security boundary, permission rules, resource capacity constraints, temporal dependencies, and security margin parameters.

[0032] Minimum distance refers to the distance metric from the predicted state of the target to the safety boundary. In some implementations, the minimum distance is used to measure the safety margin of a candidate action from the risk boundary.

[0033] Safety margin refers to a pre-set minimum safety distance threshold, used to determine whether a candidate action can be directly allowed or whether further action fine-tuning is required.

[0034] The reliability score is a stability metric calculated based on the historical execution results, out-of-bounds occurrences, rollbacks, and human intervention of a specific historical action command. A higher reliability score indicates that the corresponding historical action is more likely to be reused in subsequent similar tasks. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall system architecture.

[0036] Figure 2 This is a schematic diagram of the main process flow.

[0037] Figure 3 This is a schematic diagram of the similarity retrieval and candidate action generation process.

[0038] Figure 4 This is a schematic diagram of the process for boundary verification, action fine-tuning, and re-verification.

[0039] Figure 5 This is a schematic diagram of the process of writing back the execution result and updating the cache level.

[0040] Figure 6 This is a schematic diagram of the computing device. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the technical solutions of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0042] The system architecture, parameter values, and application scenarios described in the embodiments of this application are intended to more clearly illustrate the technical solutions provided in this application and do not constitute a limitation on the technical solutions provided in this application. Those skilled in the art will understand that equivalent substitutions or conventional adjustments made to module division, parameter configuration, formula form, or deployment location without departing from the technical concept of this application should fall within the protection scope of this application.

[0043] The candidate action safe release and closed-loop update scheme provided in this application adopts an overall technical architecture of "similarity hierarchical generation of candidate actions + boundary distance-driven safe release + execution result write-back and driving subsequent path selection updates". This overall technical architecture includes at least the following steps: task context acquisition, target state perception, feature construction, historical feature retrieval, candidate action generation, boundary verification, action release, execution result collection, and historical knowledge update, thereby forming an execution governance closed loop for action release of autonomous intelligent agents.

[0044] like Figure 1 As shown in the embodiment of this application, the candidate action safe release and closed-loop update system 100 for autonomous intelligent agents includes: a task context access layer 110, a candidate action generation layer 120, a boundary verification and action release layer 130, and a closed-loop update layer 140.

[0045] The task context access layer 110 is used to receive task context data output by the autonomous agent and collect current state data and risk constraint data of the target system. The task context data may come from an autonomous agent orchestration platform, business workflow platform, equipment operation and maintenance platform, or scheduling and control platform. The current state data may come from field sensors, business databases, execution logs, digital twin platforms, or operation monitoring interfaces. The risk constraint data may come from a permission system, rule engine, security policy library, capacity configuration library, or boundary parameter configuration interface.

[0046] The candidate action generation layer 120 includes a feature construction module 121, a similarity retrieval module 122, a historical feature library 123, a historical decision cache 124, and a candidate action generation module 125. The feature construction module 121 constructs the current feature vector based on task context data and current state data; the similarity retrieval module 122 retrieves the historical feature most similar to the current feature vector from the historical feature library 123; and the candidate action generation module 125 generates candidate action instructions by selecting from historical direct reuse, local optimization, or re-optimization of the solution path based on the target similarity.

[0047] The boundary verification and action release layer 130 includes a boundary verification sandbox 131, a distance calculation unit 132, an action adjustment unit 133, and an action release unit 134. The boundary verification sandbox 131 is used to predict the target's predicted state after executing candidate actions; the distance calculation unit 132 is used to calculate the minimum distance from the target's predicted state to the safety boundary; the action adjustment unit 133 is used to generate a corrected action instruction when the minimum distance is lower than the safety margin, and to generate a conservative action instruction when the release conditions are still not met after re-verification; the action release unit 134 is used to send the verified action instruction or the conservative action instruction to the execution interface.

[0048] The closed-loop update layer 140 includes an execution result acquisition module 141, a verification result summary module 142, and a historical knowledge update module 143. The execution result acquisition module 141 is used to collect the actual execution results of the action and the post-execution status; the verification result summary module 142 is used to summarize the verification results before release and the execution effect after release; the historical knowledge update module 143 is used to update the credibility tags, reuse priorities, cache levels, path selection threshold parameters, and action release logs in the historical feature library 123 and the historical decision cache 124, and uses the updated results for subsequent candidate action generation path determination.

[0049] In one implementation, the fields in the task context data may include at least: a task type field, a target object field, an action variable field, an action reason field, an action confidence field, and an upstream event field. The fields in the current state data may include at least: an object state field, a risk state field, a resource state field, a time state field, and a boundary state field. The fields in the risk constraint data may include at least: a permission rule field, a resource capacity limit field, a time-series dependency field, a security boundary field, and a security margin field.

[0050] In one implementation, the current feature vector It can be derived from task context subvectors Current state subvector and risk constraint subvectors The assembly is as follows, formula (1):

[0051] In formula (1), Represents the current feature vector. Represents a task context subvector. Represents the current state subvector. Represents the risk constraint subvector. , and These represent the weight coefficients of task context, current state, and risk constraints in the feature construction process, respectively. This approach allows task semantics, system state, and boundary risks to be simultaneously incorporated into the similarity retrieval process.

[0052] In one implementation, each historical sample in the historical feature library 123 includes at least the following fields: historical feature vector, historical action instruction, historical action generation path, historical boundary verification result, historical execution result, historical post-execution status, credibility score, and cache level flag. With this structure, subsequent searches can not only retrieve the historical action itself, but also whether the historical action was stable in the past, whether it frequently triggered fine-tuning, and whether it is worth prioritizing reuse.

[0053] like Figure 2 As shown, the main flow of the method provided in this application embodiment includes steps S210 to S280.

[0054] In step S210, the task context data, target system current state data, and risk constraint data generated by the autonomous agent are obtained, and the current feature vector is constructed.

[0055] In one implementation, task context data, current state data, and risk constraint data are first preprocessed. This preprocessing may include outlier removal, dimensional normalization, temporal alignment, category coding, constraint parameter standardization, and missing value imputation. For example, temperature data, load data, resource status data, and boundary parameter data generated by the same task at different time scales can be aligned using a unified sampling window before standardization.

[0056] In step S220, the similarity between the current feature vector and the historical features in the historical feature database is calculated to obtain the target similarity and determine the target historical features.

[0057] In one implementation, the similarity retrieval module 122 calculates the similarity score between the current feature vector and multiple historical features, sorts them from high to low according to the similarity score, and selects the historical features in the top K positions of the sorting result as candidate historical features, where K can be set to preset values ​​such as 3, 5, and 10. Then, the historical feature with the highest similarity score is selected as the target historical feature.

[0058] In one implementation, the similarity is calculated using cosine similarity, as shown in formula (2) below:

[0059] In formula (2), Represents historical feature vectors, This represents the similarity score between the current feature vector and the historical feature vectors.

[0060] In another implementation, Euclidean distance or Mahalanobis distance can be used to calculate the difference, and then the difference can be mapped to a similarity score. Different calculation methods can be selected for different application scenarios based on feature dimensions, feature relevance, and real-time requirements.

[0061] In step S230, the generation path of the candidate action instruction is determined based on the relationship between the target similarity and the first preset threshold and the second preset threshold.

[0062] In one implementation, the first preset threshold is higher than the second preset threshold. For example, the first preset threshold can be set to 0.85, and the second preset threshold can be set to 0.65. If the target similarity is greater than or equal to the first preset threshold, the historical action instruction corresponding to the historical features of the target is directly extracted from the historical decision cache; if the target similarity is less than the first preset threshold but greater than or equal to the second preset threshold, the historical action instruction corresponding to the historical features of the target is extracted, and local optimization is performed using the historical action instruction as the initial solution; if the target similarity is less than the second preset threshold, re-optimization is performed based on the current state data.

[0063] In one embodiment, the results output by different generation paths are uniformly encapsulated into an action tensor. The action tensor at least includes an action object identifier, action variables, an action source identifier, a target similarity score, action confidence, a generation timestamp, and a candidate action version number. Thus, the boundary verification sandbox can adopt a unified input interface for candidate actions from different sources.

[0064] In step S240, the candidate action instruction is input into the boundary verification sandbox, the target prediction state after executing the candidate action instruction is predicted according to the current effective constraint set, and the minimum distance from the target prediction state to the safety boundary is calculated.

[0065] In step S250, according to the relationship between the minimum distance and the safety margin, it is determined whether to directly release, release after action fine-tuning, or release with a conservative action.

[0066] In step S260, when the minimum distance is lower than the safety margin, an action adjustment amount is generated according to the difference between the minimum distance and the safety margin, the action variable sensitivity, the constraint coupling relationship, and a preset adjustment upper limit, a corrected action instruction is generated based on the action adjustment amount, and then the corrected action instruction is verified again.

[0067] As Figure 3 shown, in one embodiment, the similarity retrieval and the candidate action hierarchical generation process are as follows.

[0068] In a high similarity scenario, the current task context is highly similar to the current state and historical samples. At this time, the historical action instruction is directly extracted from the historical decision cache, which can significantly reduce the online solution time. For example, in the industrial operation and maintenance scenario, for the task of adjusting the equipment restart sequence that appears repeatedly, the previously verified and successfully executed action sequence can be directly reused.

[0069] In a medium similarity scenario, there are local differences between the action targets output by the autonomous agent and the historical tasks. At this time, the historical action instruction corresponding to the target historical feature is used as the initial solution to perform local optimization, which can avoid full-scale solution from scratch. For example, in the building air-conditioning control scenario, the historical action instruction can give the basic air supply volume and set temperature, while the local optimization can fine-tune the action parameters according to the current personnel density and weather changes.

[0070] In a low similarity scenario, there are not enough reusable historical action instructions in the historical feature library. At this time, a re-optimization solution is executed, and candidate action instructions are directly generated based on the current state data and the objective function. For example, in the park emergency dispatch scenario, when a new type of linkage abnormal event that has not been recorded before occurs, a re-optimization solution can be executed based on the current resource occupancy, dangerous area distribution, and task priority.

[0071] In one implementation, the local optimization path and the re-optimization solution path can share a unified objective function. For example, the objective function can be defined as formula (3):

[0072] In formula (3), This represents the cost of action changes, reflecting the magnitude of the change in the candidate action relative to historical actions or the current baseline action. This indicates a risk penalty item, reflecting the risk of the candidate action approaching or exceeding the boundary; Indicates penalties for resource conflicts or timing conflicts; Indicates the penalty for deviation from the task objective; to This represents the corresponding weighting coefficient.

[0073] In one implementation, the local optimization path uses the historical action instruction u_h as the initial solution and performs an iterative search within the local candidate action space. The local candidate action space can be generated from the neighborhood of the historical action instruction; for example, by setting forward and reverse fine-tuning step sizes for the action variables, a neighborhood action set is formed. The objective function is calculated for each action in the neighborhood action set. The action with the better objective function value is selected as the current action for the next iteration. If the number of iterations reaches a preset upper limit or the improvement of the objective function is less than a preset convergence threshold, the current action is output as a candidate action instruction.

[0074] In one implementation, the re-optimization of the solution path can be based on constructing a candidate solution space based on the range of action variable values, resource capacity constraints, temporal dependency constraints, and permission constraints. Then, optimization methods such as heuristic search, constraint programming, and integer programming are used to search for action instructions that better satisfy the objective function within the candidate solution space.

[0075] In one implementation, when the target similarity is less than a second preset threshold and there are insufficient candidate historical features in the historical feature library, similar task templates can be retrieved from the pre-trained knowledge base, template action instructions can be extracted as initial reference actions, and then the optimization solution can be re-executed based on the initial reference actions. This can improve the quality of candidate action generation in cold start or sample-scarce scenarios.

[0076] In one implementation, the historical decision cache 124 includes an L1 hot cache, an L2 stable cache, and an L3 cold cache. The L1 hot cache stores frequently hit and highly reliable historical action instructions; the L2 stable cache stores moderately hit or relatively stable historical action instructions; and the L3 cold cache stores infrequently hit but still valuable historical action instructions. This cache organization method enables candidate action generation paths in highly similar scenarios to have lower access latency.

[0077] like Figure 4 As shown, in one implementation, the boundary verification, action fine-tuning, and re-verification process is as follows.

[0078] In one implementation, constraints in the boundary verification sandbox can be divided into hard boundary constraints and soft boundary constraints. Hard boundary constraints are used to define inviolable boundaries, such as the upper limit of equipment rated load, downtime limits of critical systems, personnel safety red lines, or restricted areas. Soft boundary constraints are used to define preferred operating ranges, service quality targets, energy consumption targets, or business preference targets. For candidate actions that trigger hard boundary violation risks, adjustments should be performed first, and if the release conditions are still not met after adjustments, they should be converted to conservative action instructions. For candidate actions that only trigger soft boundary deviation risks, fine-tuning can be performed first, followed by re-verification.

[0079] In one implementation, the boundary check sandbox is based on the current state vector. Given the candidate action instruction u, the target prediction state is predicted using a preset state transition function f. Formula (4) is as follows:

[0080] In formula (4), Let u represent the current state vector, C represent the candidate action instruction, and C represent the current set of valid constraints. This indicates the predicted state of the target.

[0081] In one implementation, if the safety boundary is represented in the form of a convex polyhedron, then formula (5) is as follows:

[0082] Then the target predicted state can be The minimum distance to the safety boundary is defined by formula (6):

[0083] In formulas (5) and (6), This represents the feasible region formed by the safety boundary. Let the vector be the vector of the i-th row of matrix A. This represents the i-th boundary parameter in vector b. This represents the minimum distance from the predicted state of the target to the safety boundary. When When the value is positive, it indicates that the predicted state of the target is within the feasible region; when... When the value is close to zero, it indicates that the predicted state of the target is close to the boundary; when When the value is negative, it indicates that the predicted state of the target has exceeded the limit.

[0084] In another implementation, if the security boundary is composed of multiple heterogeneous rules, such as permission rule boundaries, time-series rule boundaries, and resource rule boundaries, then the distance metric value corresponding to each boundary category can be calculated separately. , , Then take the boundary distance with the highest risk as Formula (7) is as follows:

[0085] In formula (7), , and These represent the distance metrics corresponding to different boundary categories. Therefore, multi-rule scenarios can be uniformly transformed into an action release decision problem based on the most dangerous boundary.

[0086] In one implementation, the action variable sensitivity vector G can be obtained using a finite difference method. For the j-th action variable u_j among the candidate actions, while keeping the other action variables unchanged, a small perturbation is added to u_j. To obtain the new minimum distance Then the sensitivity of the j-th action variable to the boundary distance. It can be approximated by formula (8):

[0087] In formula (8), Indicates the perturbation step size. This represents the sensitivity of the j-th action variable to the boundary distance. This method allows for the estimation of the influence of different action variables on the boundary safety margin.

[0088] In one implementation, the motion adjustment amount It can be calculated using formula (9):

[0089] In formula (9), This represents the adjustment factor, and M represents the safety margin. Indicates the minimum distance. This represents the normalized sensitivity vector of the action variables. The smaller the minimum distance, the greater the action adjustment; different action variables bear different adjustment shares based on their sensitivity.

[0090] In one implementation, if there is a coupling relationship between multiple action variables, then the calculation can be performed... Then, coupling correction is performed, redistributing the adjustments that violate coupling relationships to other action variables. This avoids the situation where adjusting a single action variable increases the boundary distance but compromises the overall executability of the action combination.

[0091] In a specific example, assume that the building management autonomous agent generates candidate actions for a chiller plant control task, with the task objective being to reduce short-term energy consumption while maintaining comfort. The current task context includes: task type "chiller plant energy-saving regulation", target object "Area A chiller plant", and action confidence level of 0.88. The current state includes: outdoor temperature 35 degrees Celsius, area A population density 0.72, current chilled water supply temperature 7.0 degrees Celsius, current chiller load rate 0.86, and comfort risk level of medium.

[0092] In this example, the similarity scores of the current feature vector and multiple historical features in the historical feature library are ranked as follows: 0.81, 0.76, 0.62, and 0.55. Assuming the first preset threshold is 0.85 and the second preset threshold is 0.65, the target similarity is 0.81. The system selects a local optimization path and uses the historical action instructions corresponding to the target's historical features as the initial solution.

[0093] Assume that the historical action command u_h corresponding to the target historical feature includes the following three action variables: u_1 is the chilled water supply temperature setpoint, u_2 is the air volume adjustment coefficient, and u_3 is the chiller switching delay time. After local optimization, the candidate action command u_0 is obtained, where u_1 is 7.6 degrees Celsius, u_2 is 0.92, and u_3 is 2 minutes.

[0094] After inputting u_0 into the boundary verification sandbox, the target prediction state is obtained. The distances for the comfort boundary, equipment load boundary, and timing rule boundary are calculated separately to obtain... =0.16、 =0.08、 =0.11, therefore =min(0.16,0.08, 0.11)=0.08. Assuming the safety margin M is 0.12, the current candidate action does not meet the direct release condition.

[0095] Apply small perturbations to u_1, u_2 and u_3 respectively Then, the sensitivity vector G = [0.60, 0.25, 0.15] was calculated. After normalization, the value of G is taken as... =0.5, then the motion adjustment amount can be expressed as:

[0096] Based on the aforementioned action adjustment amounts, the candidate action instruction u_0 is corrected to obtain the corrected action instruction u_1. Specifically, u_1 is adjusted from 7.6 degrees Celsius to 7.72 degrees Celsius, u_2 from 0.92 to 0.915, and u_3 from 2 minutes to 2.3 minutes. After re-inputting the corrected action instruction u_1 into the boundary verification sandbox, a new minimum distance is obtained. =0.14, which is greater than the safety margin of 0.12, so the system issues the correction action instruction u_1.

[0097] In this example, the results after the actual execution of the modified action instruction u_1 show that short-term energy consumption decreased by 4.1%, the comfort index remained within the allowable range, and no alarm was triggered. This demonstrates that the action adjustment generation mechanism based on boundary distance can improve the success rate of action release while preserving the original action intent of the autonomous agent.

[0098] In one implementation, the number of re-verifications can be configured to 1, 2, or 3 times. If the safety margin requirement is still not met after exceeding the preset number of re-verifications, a conservative action instruction can be initiated, and a manual takeover request can be further triggered based on the business governance strategy. For example, in an industrial control scenario, the conservative action instruction could be "maintain the current state"; in an enterprise process scenario, the conservative action instruction could be "submit a manual approval node"; and in a scheduling scenario, the conservative action instruction could be "execute only low-risk sub-actions".

[0099] like Figure 5 As shown, in one implementation, the process of writing back the execution result and updating the cache level is as follows.

[0100] In one implementation, the actual execution result includes at least one of the following: whether the action was successfully executed, execution delay, post-execution state deviation, resource usage change, abnormal alarm information, rollback status, manual intervention status, or actual improvement value of the objective function. The boundary verification result includes at least one of the following: target similarity, action source path, minimum distance, action adjustment amount, number of re-verifications, and final release conclusion.

[0101] In one implementation, the actual execution results and boundary verification results are associated with target historical features, and the credibility labels of the target historical features are updated based on the relationship between the actual execution results and preset performance indicators. Credibility score. It can be updated according to formula (10):

[0102] In formula (10), Indicates the historical retention coefficient. This indicates the credibility score before the update. This indicates the updated credibility score. This indicates the score for evaluating the effectiveness of this action.

[0103] In one implementation, the effect evaluation score It is determined by at least three of the following categories: Action execution effect. Boundary security items and artificial intervention items For example, formula (11) can be used:

[0104] In formula (11), , and This represents the corresponding weight coefficient. If the action performs well, does not trigger an out-of-bounds error, and requires no manual intervention, then... The risk is relatively high; if the action requires rollback, triggering an alarm, or manual intervention after execution, then... Lower.

[0105] In the aforementioned building management example, we assume the original credibility score of the target's historical characteristics. It is 0.82. The score for evaluating the effectiveness of this action is set at 0.7. If we take 0.93, then the updated confidence score is:

[0106] In one implementation, when the updated confidence score is higher than a preset improvement threshold and the hit frequency of the target historical feature within a preset time window is higher than a preset high-frequency threshold, the corresponding historical action instruction is promoted to a higher level cache; when the updated confidence score is lower than a preset downgrade threshold, or the hit frequency is lower than a preset low-frequency threshold, the corresponding historical action instruction is downgraded to a lower level cache or marked as a low-confidence sample.

[0107] In one implementation, if the L1 hot cache upgrade threshold is set to 0.85 and the high-frequency threshold is set to 12 times / window, and the target historical feature in the aforementioned building management example has a hit frequency of 18 times in the most recent observation window, then the historical action instruction can be upgraded from the L2 stable cache to the L1 hot cache. This allows for the priority reuse of more stable historical action instructions in similar task contexts.

[0108] In one implementation, the write-back update includes not only updating the target's historical features but also archiving the action release log. The action release log may include at least a task context summary, target similarity, action generation path, minimum distance, safety margin, action adjustment amount, final release conclusion, actual execution result, and post-execution state deviation. By retaining the action release log, subsequent auditing, interpretable analysis, and offline model retraining can be supported; simultaneously, the action release log can also serve as the basis for subsequent threshold adaptive adjustment and path selection correction.

[0109] In one implementation, the historical knowledge update module can also adaptively adjust the first preset threshold and the second preset threshold based on the results of multiple consecutive executions. For example, when a certain type of task frequently triggers subsequent fine-tuning due to historical direct reuse paths near the first preset threshold, the first preset threshold can be appropriately increased; when a certain type of task shows stable performance in locally optimized paths within a medium similarity scenario, the medium similarity range can be appropriately expanded. The adaptively adjusted first and second preset thresholds participate in subsequent similarity-layered path determination, thereby making the candidate action generation path selection more closely reflect actual operational effects.

[0110] In one implementation, this application can be applied to autonomous intelligent agents for industrial park operation and maintenance, building management, enterprise process automation, cloud resource scheduling, and database operation and maintenance. Although the specific content of the task context, current state data, and boundary constraints differs across scenarios, a unified technical framework of "similarity-based hierarchical generation of candidate actions + boundary distance-driven action release + execution result write-back and driving subsequent path selection updates" can still be used. That is, by replacing specific state variables, objective functions, and boundary rules, this application can be adapted to different types of autonomous intelligent agent execution scenarios.

[0111] like Figure 6 As shown in the illustration, this application embodiment also provides a computing device 200, including a processor 210, a memory 220, a communication interface 230, and a bus 240. The processor 210 is used to execute program instructions stored in the memory 220 to implement the candidate action safe release and closed-loop update method in the above embodiment; the communication interface 230 is used to interact with the autonomous intelligent agent platform, the historical feature database, the execution interface, and the monitoring system.

[0112] In one embodiment, the memory 220 may include volatile memory and non-volatile memory for storing program code corresponding to task context processing logic, similarity retrieval logic, boundary verification logic, action adjustment logic, and historical knowledge update logic.

[0113] In one embodiment, this application also provides a computer-readable storage medium. The computer-readable storage medium stores program instructions that, when executed by a processor, can implement the method steps described in any of the above embodiments.

[0114] In one embodiment, this application also provides a computer program product. The computer program product includes program code that, when run on a computing device, causes the computing device to perform the method steps described in any of the above embodiments.

Claims

1. A method for safe release and closed-loop update of candidate actions for autonomous intelligent agents, characterized in that, include: The process involves acquiring task context data generated by the autonomous agent for the target system, the current state data of the target system, and risk constraint data corresponding to the target system; constructing a current feature vector based on the task context data, the current state data, and the risk constraint data; and calculating the similarity between the current feature vector and multiple historical features in the historical feature library to determine the target similarity and target historical features. The generation path of candidate action instructions is determined based on the relationship between the target similarity and a first preset threshold and a second preset threshold, wherein the first preset threshold is higher than the second preset threshold. When the target similarity is greater than or equal to the first preset threshold, historical action instructions corresponding to the historical features of the target are extracted from the historical decision cache as candidate action instructions. When the target similarity is less than the first preset threshold but greater than or equal to the second preset threshold, historical action instructions corresponding to the historical features of the target are extracted, and local optimization is performed using the historical action instructions as the initial solution to obtain candidate action instructions. When the target similarity is less than the second preset threshold, re-optimization is performed based on the current state data to obtain candidate action instructions. The candidate action instructions are input into the boundary verification sandbox, and the target prediction state after executing the candidate action instructions is predicted based on the current effective constraint set, and the minimum distance from the target prediction state to the safety boundary is calculated. When the minimum distance is greater than or equal to a preset safety margin, the candidate action instructions are used as action instructions to be issued. When the minimum distance is less than the preset safety margin, according to... The difference between the preset safety margin and the minimum distance, the sensitivity of the action variable, the constraint coupling relationship, and the preset adjustment upper limit are used to generate a corrected action instruction, which is then re-verified. When the re-verification meets the release conditions, the corrected action instruction is used as an action instruction to be issued. When the re-verification still does not meet the release conditions, a conservative action instruction is generated and used as an action instruction to be issued. The action instruction to be issued is sent to the execution interface to obtain the actual execution result and post-execution status corresponding to the action instruction to be issued. The actual execution result, the post-execution status, the target similarity, the generation path of the candidate action instruction, the boundary verification result, and the final release conclusion are associated with the target historical features and written back to at least one of the historical feature library and the historical decision cache to update the credibility score, reuse priority, cache level, and path selection threshold parameters corresponding to the target historical features. When generating subsequent candidate actions, the similarity layered path determination and action release decision are re-executed based on the updated credibility score, reuse priority, cache level, and path selection threshold parameters.

2. The method according to claim 1, characterized in that, The current feature vector is formed by concatenating the task context sub-vector, the current state sub-vector, and the risk constraint sub-vector after weighted mapping; each historical sample in the historical feature library includes at least the historical feature vector, historical action instruction, historical action generation path, historical boundary verification result, historical execution result, historical post-execution state, credibility score, and cache level marker.

3. The method according to claim 1 or 2, characterized in that, The determination of target similarity and target historical features includes: calculating the similarity score between the current feature vector and multiple historical features; sorting them from high to low according to the similarity score; selecting the historical features in the top K positions of the sorting result as candidate historical features; taking the historical feature with the highest similarity score among the candidate historical features as the target historical feature, and determining the corresponding similarity score as the target similarity.

4. The method according to claim 1, characterized in that, The candidate action instructions are uniformly encapsulated as action tensors, which at least include action object identifier, action variable, action source identifier, target similarity score, action confidence, generation timestamp, and candidate action version number. The local optimization and the re-optimization solution share a unified objective function, which at least includes an action change cost term, a risk penalty term, a resource conflict or timing conflict penalty term, and a deviation from the task objective penalty term. The local optimization includes constructing a local candidate action space using the historical action instructions as the initial solution, calculating the unified objective function for multiple neighboring actions within the local candidate action space, and outputting the current action as the candidate action instruction when a preset iteration stopping condition is met. When the target similarity is less than the second preset threshold and the candidate historical features in the historical feature library are insufficient, similar task templates are retrieved from the pre-trained knowledge base, and template action instructions corresponding to the similar task templates are extracted as initial reference actions. Re-optimization solution is then performed based on the initial reference actions.

5. The method according to claim 1, characterized in that, The constraints in the boundary verification sandbox include at least one of hard boundary constraints and soft boundary constraints. The hard boundary constraints are used to define unbreakable boundaries, and the soft boundary constraints are used to define preferred operating ranges, service quality targets, energy consumption targets, or business preference targets. The minimum distance is obtained by calculating the distance from the target prediction state to the safety boundary represented by a convex polyhedron, or by calculating the distance metrics corresponding to multiple heterogeneous boundaries and taking the distance corresponding to the most dangerous boundary as the minimum distance.

6. The method according to claim 1 or 5, characterized in that, The sensitivity of the action variable is obtained by perturbing each action variable in the candidate action and comparing the change in minimum distance before and after the perturbation; the action adjustment amount used to generate the corrected action command is determined by the difference between the preset safety margin and the minimum distance, the normalized action variable sensitivity vector, and the adjustment coefficient; when there is a coupling relationship between multiple action variables, the corrected action command is generated after coupling correction of the action adjustment amount; the number of times the re-verification is performed is 1, 2, or 3.

7. The method according to claim 1, characterized in that, When writing back to at least one of the historical feature library and the historical decision cache, an action release log is also archived; the action release log includes at least a task context summary, target similarity, action generation path, minimum distance, safety margin, action adjustment amount, final release conclusion, actual execution result, and post-execution state deviation; the credibility score is updated jointly based on the historical retention coefficient and the effect evaluation score of this action, and the effect evaluation score is determined jointly by at least the action execution effect item, the boundary safety item, and the manual intervention item.

8. The method according to claim 1 or 7, characterized in that, The historical decision cache includes an L1 hot cache, an L2 stable cache, and an L3 cold cache. When the updated confidence score is higher than a preset improvement threshold and the hit frequency of the target historical feature within a preset time window is higher than a preset high-frequency threshold, the corresponding historical action instruction is promoted to a higher-level cache. When the updated confidence score is lower than a preset downgrade threshold or the hit frequency is lower than a preset low-frequency threshold, the corresponding historical action instruction is downgraded to a lower-level cache or marked as a low-confidence sample. Furthermore, after writing back, the first preset threshold and the second preset threshold are adaptively adjusted based on the results of multiple consecutive executions, and the adaptively adjusted first preset threshold and second preset threshold are used for subsequent candidate action instruction generation path selection.

9. A candidate action safe release and closed-loop update system for autonomous intelligent agents, characterized in that, include: The feature construction module is used to obtain the task context data, current state data of the target system, and risk constraint data generated by the autonomous agent for the target system, and to construct the current feature vector; The similarity retrieval module is used to calculate the similarity between the current feature vector and multiple historical features in the historical feature database, and to determine the target similarity and the target historical feature. The candidate action generation module is used to determine the generation path of the candidate action instruction based on the relationship between the target similarity and the first preset threshold and the second preset threshold, and to generate the candidate action instruction; the boundary verification module is used to predict the target prediction state after executing the candidate action instruction based on the current effective constraint set, and to calculate the minimum distance from the target prediction state to the safety boundary. The action release module is used to issue candidate action commands when the minimum distance is greater than or equal to a preset safety margin, and to generate a corrected action command when the minimum distance is less than the preset safety margin, to re-verify the corrected action command, and to issue the corrected action command when the release conditions are met, or to generate and issue a conservative action command when the release conditions are still not met after the re-verification; the closed-loop update module is used to obtain the actual execution result and post-execution state corresponding to the issued action command, and to update the actual execution result, the post-execution state, the target similarity, the candidate action command generation path, the boundary verification result, and the release condition. The conclusion is written back to at least one of the historical feature library and the historical decision cache to update the credibility score, reuse priority, cache level, and path selection threshold parameters, and the updated results are used for subsequent candidate action generation path determination and action release decision; wherein, the boundary verification module includes a boundary verification sandbox and a distance calculation unit, the action release module includes an action adjustment unit, a re-verification unit, and a conservative action generation unit, and the closed-loop update module includes an execution result acquisition unit, a verification result summarization unit, a credibility update unit, a cache level adjustment unit, a threshold adaptive adjustment unit, and a path selection update unit.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images