A long-term goal-based AIAgent decision execution method and system

Abstract

The application relates to the technical field of artificial intelligence, and discloses an AIAgent decision execution method and system based on long-term goals, which structurize long-term goals into a hard constraint set, a trend constraint set and a deviable boundary set, construct a goal tension field based on a state feature vector, calculate a goal tension change amount corresponding to a candidate action, generate a candidate action evaluation value in combination with a risk evaluation result, determine a target action under the condition of meeting the hard constraint set, and adjust the execution intensity of the target action according to the deviable boundary set. The constraint weight parameter is updated or the candidate action set is regenerated when a preset condition is met. The long-term goals are structurized into the hard constraint set, the trend constraint set and the deviable boundary set, so that the candidate action is subjected to the calculation limitation of the long-term goal constraint before being selected, thereby avoiding the problem that the long-term goal deviates only in the execution stage or the result stage.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to an AIAgent decision-making and execution method and system based on long-term goals. Background Technology

[0002] As the application of artificial intelligence technology continues to expand in complex system control, multi-stage task planning, and autonomous decision-making scenarios, AI agents typically need to continuously perform state perception, action selection, and execution control in dynamic environments based on predetermined goals.

[0003] In existing technologies, to address the problem of long-term goal constraints, the long-term goal is typically broken down into several phased sub-goals or introduced as a termination condition or constraint into the planning process. During the task planning phase, a complete or partial execution path is generated based on the long-term goal; during the execution phase, the next action is selected based on the current state, and the planning results are locally modified as necessary. In the above technical solutions, the long-term goal mainly participates in the planning or path generation phase, while in subsequent continuous decision-making processes, action evaluation and selection are mainly based on current state information, immediate benefit indicators, or local constraints.

[0004] However, in actual operation, the inventors of this application found that the prior art cannot quantify and control the degree of deviation between the candidate action and the long-term goal constraint before the candidate action is selected. This easily leads to the deviation of the long-term goal constraint being discovered only in the execution stage or the execution result stage, making it difficult to effectively suppress the deviation of the long-term goal in the action selection stage. Summary of the Invention

[0005] To overcome the above shortcomings, this invention provides an AIAgent decision-making execution method and system based on long-term goals, aiming to improve the problem that existing technologies often lead to deviations from long-term goal constraints that can only be detected in the execution or execution result stages, making it difficult to effectively suppress deviations from long-term goals in the action selection stage.

[0006] In a first aspect, the present invention provides the following technical solution: an AIAgent decision-making execution method based on long-term goals, comprising the following steps:

[0007] S1. Obtain the current environmental state and map the current environmental state into a state feature vector through a preset state feature mapping function;

[0008] S2. Structure the preset long-term goals into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries;

[0009] S3. Based on the set of hard constraints, the set of trend constraints, and the set of deviable boundaries, calculate the target tension value corresponding to the state feature vector, thereby constructing the target tension field;

[0010] S4. Generate a set of candidate actions based on the current environment state;

[0011] S5. Perform state prediction for each candidate action in the candidate action set to obtain the corresponding predicted state, and map the predicted state to a predicted state feature vector.

[0012] S6. Based on the predicted state feature vector, calculate the target tension change corresponding to each candidate action, and construct the candidate action evaluation value in combination with the risk assessment results.

[0013] S7. Under the condition of satisfying the set of hard constraints, determine the target action from the set of candidate actions according to the evaluation value of the candidate actions;

[0014] S8. Based on the relationship between the state feature vector and the set of deviable boundaries, adjust the execution intensity of the target action and execute the adjusted target action;

[0015] S9. Perform consistency detection on the target tension values ​​corresponding to multiple consecutive time points. When the detection results meet the preset conditions, execute the target consistency reconstruction step.

[0016] Preferably, in step S2, the step of structuring the preset long-term goal into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries includes:

[0017] The long-term goal is broken down into multiple target constraints;

[0018] Based on the target constraint terms, the corresponding hard constraint conditions, trend constraint conditions, and deviation boundary conditions are determined respectively.

[0019] The hard constraints are represented as a set of hard constraint functions, the trend constraints are represented as a set of trend constraint functions, and the deviable boundary conditions are represented as a set of deviable boundary functions.

[0020] Furthermore, the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions all use the state feature vector as the input variable.

[0021] Preferably, in step S3, the step of calculating the target tension value corresponding to the state feature vector, thereby constructing the target tension field, includes:

[0022] Obtain the function output results of the state feature vector in the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions, respectively;

[0023] The violation degree of each function output is calculated to obtain the corresponding hard constraint violation degree, trend violation degree, and boundary violation degree;

[0024] The hard constraint tension, trend tension, and boundary tension are calculated based on the hard constraint violation degree, trend violation degree, and boundary violation degree, respectively.

[0025] The hard constraint tension, trend tension, and boundary tension are then combined to obtain the target tension value.

[0026] Preferably, the step of calculating the violation degree of each function output includes:

[0027] Determine if the output of the constraint function is greater than zero;

[0028] When the output of the constraint function is less than or equal to zero, the corresponding violation degree is set to zero;

[0029] When the output of the constraint function is greater than zero, the corresponding violation degree is determined based on the magnitude of the output value.

[0030] Preferably, in step S5, the step of predicting the state of each candidate action in the candidate action set includes:

[0031] Use the current environment state and candidate actions as input parameters;

[0032] Based on the input parameters, a one-step state prediction model is invoked to generate the predicted state;

[0033] The predicted state is then converted into a predicted state feature vector using the state feature mapping function.

[0034] Preferably, in step S6, the step of constructing candidate action evaluation values ​​based on the risk assessment results includes:

[0035] Calculate the target tension value corresponding to the predicted state feature vector and the target tension value corresponding to the current state feature vector, respectively.

[0036] The target tension change is determined based on the two target tension values ​​mentioned above;

[0037] Obtain the risk assessment results corresponding to the candidate actions;

[0038] The candidate action evaluation value is obtained by combining the target tension change with the risk assessment result.

[0039] Preferably, in step S7, the step of determining the target action from the candidate action set based on the candidate action evaluation value includes:

[0040] Based on the predicted state corresponding to the candidate action, determine whether each candidate action satisfies the set of hard constraints.

[0041] The candidate actions that satisfy the set of hard constraints are taken as the set of optional actions;

[0042] Then, the candidate action with the smallest evaluation value is selected from the set of optional actions as the target action.

[0043] Preferably, in step S8, the step of adjusting the execution intensity of the target action and executing the adjusted target action includes:

[0044] Based on the state feature vector, calculate its function output in the set of deviable boundary functions;

[0045] The corresponding execution intensity parameter is determined based on the function output.

[0046] The target action is then performed based on the execution intensity parameter.

[0047] Preferably, in step S9, the step of performing the target consistency reconstruction step when the detection result meets the preset conditions includes:

[0048] Obtain the target tension values ​​corresponding to multiple consecutive time points;

[0049] Construct a tension variation sequence based on the target tension value;

[0050] Determine whether the preset reconstruction conditions are met based on the tension change sequence;

[0051] When the preset reconstruction conditions are met, the weight parameters corresponding to the hard constraint set, trend constraint set, and deviation boundary set are updated, or a new candidate action set is generated.

[0052] Secondly, the present invention provides the following technical solution: an AIAgent decision-making and execution system based on long-term goals, comprising:

[0053] The state acquisition and feature mapping module is used to acquire the environment state and generate state feature vectors;

[0054] The long-term goal structuring module is used to structure long-term goals into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries.

[0055] The target tension field construction module is used to construct the target tension field based on the set of hard constraints, the set of trend constraints, and the set of deviable boundaries.

[0056] The candidate action generation and prediction module is used to generate a set of candidate actions and obtain the corresponding prediction state;

[0057] The action evaluation and selection module is used to calculate the evaluation function of candidate actions and select the target action under the premise of satisfying the set of hard constraints.

[0058] The execution adjustment module is used to adjust the execution intensity of the target action based on the determination result of the set of possible deviations from the boundary.

[0059] The target consistency detection and reconstruction module is used to detect the consistency of the target tension state and trigger the target consistency reconstruction process when the conditions are met.

[0060] The present invention has the following beneficial effects:

[0061] 1. This invention structures the long-term goal into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries, and calculates the target tension value based on the above sets at each moment. The change in target tension is directly introduced into the candidate action evaluation process, so that the candidate action is subject to the calculation limitation of the long-term goal constraint before it is selected, thereby avoiding the problem of discovering the deviation of the long-term goal only in the execution stage or the result stage.

[0062] 2. This invention calculates the violation degree of the output results of hard constraint function, trend constraint function and deviable boundary function in a unified manner, and constructs hard constraint tension, trend tension and boundary tension respectively, and then forms a unified target tension value. The candidate action evaluation value is constructed by combining the target tension change and the risk assessment result, so that constraints of different sources and types participate in action ranking under the same evaluation framework, thereby achieving consistent action screening results in scenarios with multiple long-term constraints and risk constraints.

[0063] 3. This invention constructs a tension change sequence based on the target tension values ​​at multiple consecutive moments, and performs consistency detection based on the tension change sequence. When the preset reconstruction conditions are met, the constraint weight parameters are updated or a candidate action set is regenerated. This enables the system to identify the cumulative state of long-term target constraint deviation and readjust subsequent decision inputs during multi-step execution, thereby suppressing the continuous deviation of long-term target constraints during continuous execution. Attached Figure Description

[0064] Figure 1 This is a flowchart of an AIAgent decision-making and execution method based on long-term goals proposed in this invention;

[0065] Figure 2 This is an architecture diagram of an AIAgent decision-making and execution system based on long-term goals proposed in this invention. Detailed Implementation

[0066] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0067] Reference Figure 1 This invention provides an AIAgent decision-making execution method based on long-term goals, comprising the following steps:

[0068] S1. Obtain the current environment state and map the current environment state into a state feature vector through a preset state feature mapping function;

[0069] S2. Structure the preset long-term goals into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries;

[0070] S3. Based on the set of hard constraints, the set of trend constraints, and the set of deviable boundaries, calculate the target tension value corresponding to the state feature vector, thereby constructing the target tension field;

[0071] S4. Generate a set of candidate actions based on the current environment state;

[0072] S5. Perform state prediction for each candidate action in the candidate action set, obtain the corresponding predicted state, and map the predicted state to a predicted state feature vector.

[0073] S6. Based on the predicted state feature vector, calculate the target tension change corresponding to each candidate action, and construct the candidate action evaluation value in combination with the risk assessment results.

[0074] S7. Under the condition of satisfying the set of hard constraints, determine the target action from the set of candidate actions based on the evaluation value of the candidate actions;

[0075] S8. Based on the relationship between the state feature vector and the set of deviable boundaries, adjust the execution intensity of the target action and execute the adjusted target action.

[0076] S9. Perform consistency detection on the target tension values ​​corresponding to multiple consecutive time points. When the detection results meet the preset conditions, execute the target consistency reconstruction step.

[0077] Specifically, in this embodiment, the current environmental state is used as input, and a unified state feature vector expression is formed through state feature mapping. This state feature vector serves as the unified calculation object for long-term goal constraint calculation, candidate action evaluation, and execution adjustment. By transforming the long-term goal into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries, and calculating the goal tension value based on the set on the state feature vector at each decision moment, the long-term goal participates in the prediction and evaluation of candidate actions and the determination of target actions in the form of constraint functions and tension calculations.

[0078] In the candidate action generation and prediction stage, feature mapping is performed on the predicted states corresponding to candidate actions, and the corresponding change in target tension is calculated. This incorporates the changes in candidate actions within the long-term target constraint space along with the risk assessment results into the construction of candidate action evaluation values. In the target action determination stage, the predicted states are screened based on whether they satisfy the hard constraint set. From the candidate actions that satisfy the hard constraint set, the target action is determined based on the candidate action evaluation values. In the target action execution stage, the execution intensity parameter is determined by the constraint output results of the state feature vector within the set of deviable boundaries, and the target action is executed based on the execution intensity parameter.

[0079] During continuous execution, consistency detection is performed on the target tension values ​​corresponding to multiple consecutive time points, and the target consistency reconstruction step is executed when the preset reconstruction conditions are met. The constraint weight parameters or candidate action set are adjusted, thereby forming a complete decision-making and execution process that runs through state acquisition, candidate action evaluation, action execution and constraint reconstruction.

[0080] Furthermore, in step S2, the step of structuring the preset long-term goal into a set of hard constraints, a set of trend constraints, and a set of deviation boundaries includes:

[0081] Break down long-term goals into multiple target constraints;

[0082] Based on the target constraint terms, determine the corresponding hard constraints, trend constraints, and deviation boundary conditions respectively;

[0083] Hard constraints are represented as a set of hard constraint functions, trend constraints as a set of trend constraint functions, and deviable boundary conditions as a set of deviable boundary functions.

[0084] Furthermore, the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions all use the state feature vector as the input variable.

[0085] Specifically, in this embodiment, the long-term goal is input into the system in the form of structured goal description data. The long-term goal includes at least goal identification information, goal constraint type identification information, and goal constraint parameter information. The system first decomposes the long-term goal into several independent goal constraint items based on the goal identification information. Each goal constraint item corresponds to a computable constraint rule. Each goal constraint item includes at least the following fields: constraint type field, constraint effect dimension field, and constraint parameter field. The constraint type field indicates whether the goal constraint item is a hard constraint, a trend constraint, or a deviable boundary condition; the constraint effect dimension field indicates one or more component indices in the state feature vector corresponding to the goal constraint item; and the constraint parameter field describes the numerical boundary, threshold range, or reference target value corresponding to the goal constraint item.

[0086] During the target constraint classification phase, the system categorizes target constraints into three sets based on the constraint type field: hard constraint set, trend constraint set, and deviable boundary condition set. For target constraints belonging to the hard constraint set, the system generates corresponding hard constraint functions according to their constraint parameter fields; for target constraints belonging to the trend constraint set, the system generates corresponding trend constraint functions according to their constraint parameter fields; and for target constraints belonging to the deviable boundary condition set, the system generates corresponding deviable boundary functions according to their constraint parameter fields.

[0087] In this implementation, all constraint functions uniformly use the state feature vector as the input variable. Let the current state feature vector be... ;

[0088] The set of hard constraint functions is represented as: ;

[0089] The set of trend constraint functions is represented as follows: ;

[0090] The set of deviable boundary functions is represented as: ;

[0091] Each hard constraint function, trend constraint function, and deviable boundary function is determined by the constraint action dimension field and constraint parameter field in the corresponding target constraint term, and its input variable is the state feature vector x of the same dimension.

[0092] In the specific construction process, for each target constraint term, the system first extracts the corresponding component or subset of components from the state feature vector based on the constraint action dimension field, and then constructs the corresponding constraint expression based on the constraint parameter field. When the constraint action dimension field indicates multiple components, intermediate constraint quantities are generated for the multiple components according to a preset combination method, and then the corresponding constraint functions are constructed based on the intermediate constraint quantities. Among them, the hard constraint function is used to describe the non-crossing boundary conditions of the state feature vector in the corresponding constraint action dimension, the trend constraint function is used to describe the reference direction or reference interval constraint of the state feature vector in the corresponding constraint action dimension, and the deviation boundary function is used to describe the allowable deviation interval boundary of the state feature vector in the corresponding constraint action dimension.

[0093] After constructing the sets of hard constraint functions, trend constraint functions, and deviable boundary functions, the system uniformly registers and manages these three sets of constraint functions and establishes a mapping table between constraint functions and state feature vector component indices. This ensures that each constraint function can be invoked based on the same state feature vector input during subsequent target tension value calculations. Simultaneously, to ensure consistency in constraint expression, the outputs of the hard constraint function sets, trend constraint function sets, and deviable boundary function sets are all standardized into scalar form, enabling unified processing of constraint results from different sources and with different constraint dimensions during subsequent violation and tension calculations.

[0094] Through the above steps, the original long-term objective is transformed into a set of hard constraint functions, a set of trend constraint functions, and a set of deviable boundary functions oriented towards the state feature space, providing a unified constraint input structure for subsequent objective tension value calculation.

[0095] Furthermore, in step S3, the step of calculating the target tension value corresponding to the state feature vector, thereby constructing the target tension field, includes:

[0096] Obtain the function output results of the state feature vector in the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions, respectively;

[0097] The violation degree of each function output is calculated to obtain the corresponding hard constraint violation degree, trend violation degree, and boundary violation degree;

[0098] Calculate the hard constraint tension, trend tension, and boundary tension based on the hard constraint violation degree, trend violation degree, and boundary violation degree, respectively;

[0099] The hard constraint tension, trend tension, and boundary tension are combined to obtain the target tension value.

[0100] The steps for calculating the violation degree of each function's output include:

[0101] Determine if the output of the constraint function is greater than zero;

[0102] When the output of the constraint function is less than or equal to zero, the corresponding violation degree is set to zero;

[0103] When the output of the constraint function is greater than zero, the corresponding degree of violation is determined based on the magnitude of the output value.

[0104] Specifically, in this embodiment, the calculation of the target tension value uses the state feature vector as the only input object, and is based on the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions formed in step S2.

[0105] Within a single target tension value calculation cycle, the system first reads the current state feature vector and, according to the registration order of the constraint function set, inputs the state feature vector into each hard constraint function, trend constraint function, and deviable boundary function to obtain the corresponding function output results. These function output results are uniformly cached as constraint output vectors, retaining the correspondence between constraint function identifiers and output values ​​for subsequent violation calculations and tension component calculations. During the violation calculation phase, the system processes the output result of each constraint function according to a unified judgment rule. For any constraint function output result z, the system determines the violation degree based on the following rules:

[0106] ;

[0107] When the constraint function output is less than or equal to zero, the corresponding violation degree is recorded as zero; when the constraint function output is greater than zero, the output result is directly used as the violation degree input value for subsequent calculations. After completing the violation degree calculation, the system classifies and summarizes different types of violations, forming sets of hard constraint violations, trend violations, and boundary violations.

[0108] In the hard constraint tension calculation phase, the system weights each hard constraint violation with its corresponding weight parameter and accumulates the weighted results to obtain the hard constraint tension. This weighted accumulation process uses the violation degree of each hard constraint function as input and calculates according to the weight parameters corresponding one-to-one with each hard constraint function. In the trend tension calculation phase, the system weights each trend constraint violation with its corresponding trend constraint weight parameter and accumulates the weighted results to obtain the trend tension. In the boundary tension calculation phase, the system weights each deviation from the boundary with its corresponding deviation from the boundary weight parameter and accumulates the weighted results to obtain the boundary tension.

[0109] In this embodiment, the hard constraint tension, trend tension, and boundary tension are calculated as follows:

[0110] ;

[0111] ;

[0112] ;

[0113] in, This is the output of the i-th hard constraint function;

[0114] This is the output of the j-th trend constraint function;

[0115] For the first The output of a boundary function that can deviate from the boundary function;

[0116] These are the weight parameters corresponding to each constraint function.

[0117] After completing the calculation of the three types of tension, the system combines the hard constraint tension, trend tension, and boundary tension to obtain the target tension value corresponding to the state feature vector:

[0118] ;

[0119] The mapping relationship between state feature vectors and target tension values ​​constitutes the target tension field. In actual implementation, the system maintains an independent weight parameter storage unit for each constraint function and establishes a mapping table between constraint function identifiers and weight parameters, enabling the corresponding weight parameters to be updated for a specified constraint function during the target consistency reconstruction phase. Simultaneously, after the target tension value is calculated, the system writes the target tension value and the corresponding state feature vector together into a time series cache for use in the subsequent consistency detection phase to construct the tension change sequence.

[0120] Through the above-described target tension value calculation process, different types of long-term target constraints can participate in state evaluation with a unified violation degree and tension calculation method. Hard constraints, trend constraints, and deviation boundary constraints are combined under the same target tension calculation framework to form a target tension field for subsequent candidate action evaluation and consistency detection, thereby realizing the computable expression effect of long-term target constraints in the state space.

[0121] Furthermore, in step S5, the step of predicting the state for each candidate action in the candidate action set includes:

[0122] Use the current environment state and candidate actions as input parameters;

[0123] The predicted state is generated by calling a one-step state prediction model based on the input parameters.

[0124] The predicted state is then converted into a predicted state feature vector using a state feature mapping function.

[0125] Specifically, in this embodiment, after the candidate action set is formed in step S4, it is written into the candidate action buffer, and state prediction processing is performed sequentially according to the order in which the candidate actions are generated. When performing state prediction, the system combines the current environment state with a single candidate action to form a prediction input pair, and performs a state prediction process once for each candidate action.

[0126] In this embodiment, the one-step state prediction model is used to describe the determination or estimation relationship of the environment state at the next moment, given the current environment state and candidate actions. The one-step state prediction model consists of a system operation rule model, an environment simulation model, an interface query model, or a state transition mapping model built based on historical interaction data. Its input fields include at least all or part of the fields of the current environment state and the action parameter fields corresponding to the candidate actions. Let the current environment state be... The i-th candidate action is represented as The one-step state prediction model is denoted as Then, for each candidate action, the corresponding predicted state is generated as follows: .

[0127] The state feature mapping function processes each field in the predicted state according to the same feature extraction rules as the current state, including field selection, numerical normalization, discrete field encoding, and missing field imputation. This ensures the consistency between the predicted state feature vector and the current state feature vector in terms of dimension, component meaning, and component order. Simultaneously, after generating the predicted state feature vector, the system performs consistency checks, including dimensional consistency checks, numerical validity checks, and field correspondence checks. If a check fails, the corresponding candidate action is marked as unsuitable for subsequent evaluation calculations. An index mapping relationship is established between the predicted state and the predicted state feature vector and the corresponding candidate action, and this mapping is written to the prediction result cache for direct use in subsequent target tension change calculations and candidate action evaluation value construction stages.

[0128] Through the above state prediction processing flow, each candidate action corresponds to a set of predicted state feature vectors that are consistent with the current state feature vector structure before entering the calculation of the target tension change. This provides a unified data input basis for subsequent calculation of the target tension value and target tension change based on the predicted state feature vectors, and realizes the computable predictive effect of candidate actions in the state space.

[0129] Furthermore, in step S6, the step of constructing candidate action evaluation values ​​based on the risk assessment results includes:

[0130] Calculate the target tension value corresponding to the predicted state feature vector and the target tension value corresponding to the current state feature vector, respectively.

[0131] The target tension change is determined based on the two target tension values ​​mentioned above;

[0132] Obtain the risk assessment results corresponding to the candidate actions;

[0133] The candidate action evaluation value is obtained by combining the target tension change and the risk assessment results.

[0134] Specifically, in this embodiment, the construction of candidate action evaluation values ​​uses the predicted state feature vector formed in S5 as input, and reuses the target tension value calculation process established in S3 to calculate the target tension value for each predicted state feature vector. For the current decision moment, the system first reads the target tension value corresponding to the current state feature vector from the cache and saves it as the baseline tension value; then, for each candidate action corresponding to the predicted state feature vector, the corresponding predicted target tension value is calculated according to the same constraint function call order, violation calculation rules, and tension combination rules as in S3.

[0135] Let the current state feature vector be The predicted state feature vector corresponding to the i-th candidate action is Then the target tension value in the current state and the target tension value in the predicted state are respectively: .

[0136] After completing the above target tension value calculation, the system calculates the corresponding target tension change for each candidate action. The calculation method is the difference between the predicted target tension value and the current target tension value:

[0137] ;

[0138] During the risk assessment result acquisition phase, for each candidate action, the system invokes a pre-defined risk assessment rule set to calculate the risk item for the candidate action and its corresponding current environmental state. The risk assessment rule set consists of several risk assessment rules, each corresponding to a computable risk item, and generates a corresponding risk output value based on the candidate action parameters, the current environmental state field, and system operational constraints. The output results of each risk assessment rule are uniformly normalized to a scalar form and then weighted and accumulated to form the risk assessment result corresponding to the candidate action. A correspondence is established between the risk assessment result and the candidate action, and the result is written to the risk result cache.

[0139] During the candidate action evaluation value construction phase, the system performs a combined calculation on the target tension change and risk assessment result corresponding to each candidate action. Let the combined weight parameter be... and The candidate action evaluation value is calculated as follows:

[0140] ;

[0141] in, Indicates the current environmental state Below, the risk assessment result corresponding to the i-th candidate action.

[0142] When constructing candidate action evaluation values, the validity of the target tension change and risk assessment results are verified separately, including numerical validity detection and outlier detection. If there are cases where calculation is impossible or outliers are found, the corresponding candidate action is marked as not eligible to participate in subsequent target action determination steps. Simultaneously, to ensure the repeatability and consistency of the evaluation value construction process, the system records the combined weight parameters used, the target tension change involved in the combination, the risk assessment results, and the candidate action identifiers together. This information is used for backtracking calculations of the evaluation process during subsequent consistency checks and reconstruction phases.

[0143] Through the above evaluation value construction process, candidate actions are simultaneously incorporated with the target tension change and risk assessment results under the same evaluation calculation framework, forming a unified candidate action evaluation value. This enables candidate actions to have a calculable ranking effect based on long-term target constraint changes and risk constraints in the subsequent target action determination stage.

[0144] Furthermore, in step S7, the step of determining the target action from the candidate action set based on the candidate action evaluation value includes:

[0145] Based on the predicted state corresponding to the candidate action, determine whether each candidate action satisfies the set of hard constraints.

[0146] Candidate actions that satisfy the set of hard constraints are used as the set of optional actions;

[0147] Then, select the candidate action with the smallest evaluation value from the set of available actions as the target action.

[0148] Specifically, in this embodiment, the target action determination process is executed after the candidate action evaluation values ​​are constructed in step S6. The system first reads the predicted state feature vector corresponding to each candidate action from the prediction result cache, and then performs hard constraint determination on each predicted state feature vector according to the registration order of the hard constraint functions in the hard constraint function set. Let the predicted state feature vector corresponding to the i-th candidate action be... Let the set of hard constraint functions be... For each candidate action, the system calculates sequentially: Furthermore, based on the hard constraint determination rules, the output results of each hard constraint function are uniformly determined. A candidate action is determined to satisfy the set of hard constraints if and only if the output results of all hard constraint functions corresponding to the candidate action satisfy the non-positive condition.

[0149] The system maintains an independent hard constraint determination flag for each candidate action. When the output of any hard constraint function is greater than zero, the corresponding candidate action is immediately marked as not satisfying the hard constraint set, thus avoiding further hard constraint calculations for invalid candidate actions. After completing the hard constraint determination for all candidate actions, the system extracts all candidate actions that satisfy the hard constraint set and constructs an optional action set. If the optional action set is empty, the system enters a preset exception branch process, either regenerating the candidate action set or entering the trigger judgment phase of the target consistency reconstruction process.

[0150] After the set of optional actions is constructed, the system reads the corresponding candidate action evaluation values ​​from the set and sorts the candidate action evaluation values. Let the evaluation value of the i-th candidate action be... The system compares the evaluation values ​​in ascending order and selects the candidate action with the lowest evaluation value as the target action for this decision cycle.

[0151] ;

[0152] in, This represents the set of optional actions consisting of candidate actions that satisfy the set of hard constraints.

[0153] When multiple candidate actions have the same minimum candidate action evaluation value, the system performs a secondary judgment based on the generation order of the candidate actions in the candidate action set or a preset priority identifier, thereby uniquely determining the target action. After the target action is determined, the system writes the target action identifier, the corresponding candidate action evaluation value, and the hard constraint judgment result into the decision record cache for subsequent execution adjustment and consistency detection process calls.

[0154] Through the above-described target action determination process, all candidate actions participating in the final decision are screened by a set of hard constraints before entering the ranking comparison. Within the range of candidate actions that satisfy the set of hard constraints, the target action is selected based on the evaluation value of the candidate actions. This ensures that the target action is determined based on the evaluation value calculation results, provided that the constraints are not violated.

[0155] Furthermore, in step S8, the steps of adjusting the execution intensity of the target action and executing the adjusted target action include:

[0156] Based on the state feature vector, calculate its function output in the set of deviable boundary functions;

[0157] Determine the corresponding execution intensity parameters based on the function output;

[0158] And execute the target action based on the execution intensity parameter.

[0159] Specifically, in this embodiment, after the target action is determined in S7, it is not executed directly in a fixed manner, but first enters the execution intensity adjustment stage. This stage uses the current state feature vector as input and the set of deviable boundary functions formed in S2 as constraints to adjust the execution method of the target action. During the execution intensity adjustment process, the state feature vector is first read from the current state feature vector buffer, and then, according to the registration order of each function in the set of deviable boundary functions, the state feature vector is input one by one into each deviable boundary function to obtain the corresponding function output. The function output is used to characterize the degree of deviation of the current state feature vector under the corresponding deviable boundary constraints.

[0160] Let the current state feature vector be Let the set of deviable boundary functions be... The system calculates the following respectively: The output results of each function are processed uniformly and used as the input for deviation judgment.

[0161] To facilitate unified adjustment of multidimensional deviations, the system merges the outputs of each deviable boundary function and calculates the overall deviation corresponding to the current state. The overall deviation is determined by the result with the highest violation among the outputs of each deviable boundary function, and its calculation method is as follows:

[0162] ;

[0163] in, This is a function for calculating the degree of violation.

[0164] After determining the overall deviation, the system calculates the execution intensity parameter based on the overall deviation. The execution intensity parameter is a scalar parameter used to adjust the execution amplitude, execution frequency, or execution rate of the target action. The execution intensity parameter is obtained by mapping the overall deviation, and its calculation form is as follows:

[0165] ;

[0166] in, These are the preset adjustment parameters.

[0167] During execution, the system binds the execution intensity parameter to the target action. This binding method manifests as scaling parameters related to the execution magnitude in the target action, adjusting the execution frequency of the target action, or limiting the execution time window of the target action. The specific binding method is configured based on the data structure of the target action, but is controlled by the execution intensity parameter.

[0168] After the adjustment is completed, the system sends the target action instruction to the execution interface according to the adjusted execution parameters and receives the corresponding execution receipt information to record the execution result.

[0169] In this embodiment, the execution intensity adjustment process does not change the action type of the target action, but only adjusts the execution parameters to make the target action execute with different execution intensities under different deviable boundary states.

[0170] Furthermore, in step S9, when the detection result meets the preset conditions, the steps for performing the target consistency reconstruction step include:

[0171] Obtain the target tension values ​​corresponding to multiple consecutive time points;

[0172] Construct a tension variation sequence based on the target tension value;

[0173] Determine whether the preset reconstruction conditions are met based on the tension change sequence;

[0174] When the preset reconstruction conditions are met, the weight parameters corresponding to the hard constraint set, trend constraint set, and deviation boundary set are updated, or the candidate action set is regenerated.

[0175] Specifically, in this embodiment, the target consistency reconstruction process performs a detection and judgment once after each decision cycle, using the target tension value calculated and cached by S3 as the input data source. The system maintains a continuous time window cache for the target tension values ​​to store the target tension values ​​corresponding to the most recent multiple decision times. Let the current decision time be ttt, the target tension value sequence stored in the cache is represented as:

[0176] ;

[0177] Where H is the preset time window length parameter.

[0178] After acquiring the target tension values ​​at multiple consecutive time points, the system constructs a tension change sequence in chronological order. This tension change sequence is directly generated from the time series of the target tension values ​​and is used for subsequent trend and cumulative change calculations.

[0179] In this embodiment, the system constructs tension statistics based on the tension change sequence to describe the overall change in the long-term target constraint state at the current stage. Specifically, the system calculates the moving average of the target tension value within a time window and further calculates the difference between the moving averages of adjacent time windows.

[0180] The moving average is defined as: ;

[0181] The change in tension is defined as: ;

[0182] When determining whether the preset reconstruction conditions are met, the system compares the sliding average and the tension change with the corresponding preset threshold parameters. When the sliding average exceeds the preset upper limit threshold for tension, or the tension change exceeds the preset change threshold, the system determines that the target consistency reconstruction conditions are met.

[0183] After the preset reconstruction conditions are met, the system enters the target consistency reconstruction process.

[0184] In one refactoring implementation, the system updates the weight parameters corresponding to the hard constraint set, trend constraint set, and deviable boundary set. Specifically, the system uses the violation degree of each constraint function at the current moment as input and adjusts the weight parameters of each constraint function item by item. For any hard constraint function, trend constraint function, or deviable boundary function, the weight parameter update rule is to adjust the original weight parameters proportionally based on its current violation degree, and write the updated weight parameters back to the weight parameter storage area so that they can take effect directly in the subsequent target tension value calculation.

[0185] In another refactoring implementation, the system, while keeping the original weight parameters unchanged, re-triggers the candidate action set generation process and reconstructs the candidate action set based on the current environment state. The regenerated candidate action set directly enters the S5 state prediction processing flow, replacing the original candidate action set in the subsequent candidate action evaluation and target action determination process.

[0186] After reconstruction is completed, the system clears the target tension cache data used in the current time window and starts recording the target tension values ​​corresponding to subsequent decision cycles again, so as to avoid the tension sequence before reconstruction from affecting the detection results after reconstruction.

[0187] Example 2: Refer to Figure 2 In a second embodiment of the present invention, the present invention provides an AIAgent decision execution system based on long-term goals, comprising:

[0188] The state acquisition and feature mapping module is used to acquire the environment state and generate state feature vectors;

[0189] The long-term goal structuring module is used to structure long-term goals into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries.

[0190] The target tension field construction module is used to construct the target tension field based on the set of hard constraints, the set of trend constraints, and the set of deviable boundaries.

[0191] The candidate action generation and prediction module is used to generate a set of candidate actions and obtain the corresponding prediction state;

[0192] The action evaluation and selection module is used to calculate the evaluation function of candidate actions and select the target action under the premise of satisfying the set of hard constraints.

[0193] The execution adjustment module is used to adjust the execution intensity of the target action based on the determination result of the set of possible deviations from the boundary.

[0194] The target consistency detection and reconstruction module is used to detect the consistency of the target tension state and trigger the target consistency reconstruction process when the conditions are met.

[0195] Specifically, the state acquisition and feature mapping module connects to the external environment interface to receive the current environmental state data and write it into the state buffer. This module performs field selection, numerical normalization, discrete field encoding, and missing field imputation on each state field in the environmental state according to pre-configured state feature mapping rules, generating a state feature vector, which is then stored in the feature buffer.

[0196] The long-term goal structuring module is connected to the goal configuration storage unit. It reads long-term goal description data from the goal configuration storage unit, parses the long-term goal description data, and decomposes the long-term goal into multiple goal constraint terms. Based on the constraint type identifier carried in each goal constraint term, the goal constraint terms are classified into hard constraints, trend constraints, and deviable boundary conditions. Based on the constraint parameters in each goal constraint term, a set of hard constraint functions, a set of trend constraint functions, and a set of deviable boundary functions are constructed.

[0197] The long-term goal structuring module writes three sets of constraint functions and their corresponding function identifiers into the constraint function management table for subsequent modules to call uniformly.

[0198] The target tension field construction module is connected to the feature buffer and the constraint function management table. It receives the current state feature vector and, according to the function order registered in the constraint function management table, sequentially calls the hard constraint function set, the trend constraint function set, and the deviation boundary function set to perform constraint calculations on the state feature vector and generate the corresponding constraint function output results.

[0199] This module further performs violation calculation on the output results of each constraint function, and calculates hard constraint tension, trend tension and boundary tension based on the corresponding weight parameters, and combines the three types of tension to generate the target tension value.

[0200] The target tension value and the corresponding state feature vector timestamp are written together into the tension buffer as input data for subsequent candidate action evaluation and consistency detection.

[0201] The candidate action generation and prediction module connects to the environment interface and action template library, generates a set of candidate actions based on the current environment state, and assigns a unique action identifier to each candidate action.

[0202] After generating the candidate action set, this module calls the one-step state prediction model based on the current environment state and the corresponding candidate action to generate a predicted state for each candidate action. The predicted state is then input into the same feature mapping rule used by the state feature mapping module to generate a predicted state feature vector.

[0203] Each predicted state feature vector is mapped to its corresponding candidate action identifier and written to the prediction result cache.

[0204] The action evaluation and selection module is connected to the prediction result cache, tension cache, constraint function management table, and risk assessment rule base.

[0205] For each predicted state feature vector in the prediction result cache, this module calls the same tension calculation process in the target tension field construction module to obtain the target tension value corresponding to the predicted state, and calculates the change in target tension by combining it with the target tension value corresponding to the current state.

[0206] At the same time, the module calculates the risk based on the current environmental state and candidate action parameters according to the risk assessment rule base, and generates the corresponding risk assessment results.

[0207] After obtaining the target tension change and risk assessment results, the module performs combined calculations to generate candidate action evaluation values. After completing the hard constraint set determination, the module selects the candidate action with the smallest evaluation value from the candidate actions that satisfy the hard constraint set as the target action.

[0208] The execution adjustment module is connected to the feature cache and the set of deviable boundary functions in the constraint function management table.

[0209] This module receives the current state feature vector, calculates the output of the state feature vector in each deviable boundary function, determines the overall deviation based on each output, and generates execution intensity parameters based on the overall deviation.

[0210] The execution intensity parameter is bound to the action parameter in the target action. The execution amplitude, execution frequency or execution rate parameter corresponding to the target action is adjusted, and the adjusted target action is sent to the external execution interface.

[0211] The target consistency detection and reconstruction module is connected to the tension buffer, constraint function management table and candidate action generation module.

[0212] This module periodically reads the target tension values ​​corresponding to multiple consecutive moments in the tension buffer, constructs a tension change sequence, calculates tension statistics based on the tension change sequence, and compares them with preset reconstruction conditions for judgment.

[0213] When the preset reconstruction conditions are met, the module performs update operations on the weight parameters corresponding to the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions, or sends a regeneration instruction to the candidate action generation and prediction module to regenerate the candidate action set based on the current environmental state.

[0214] In this embodiment, the modules transmit state feature vectors, predicted state feature vectors, target tension values, candidate action evaluation values, execution intensity parameters, and constraint weight parameters through an internal data bus, realizing a complete system processing flow from environmental state acquisition, long-term target structuring, target tension calculation, candidate action prediction and evaluation, action execution adjustment, and consistency detection and reconstruction.

[0215] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An AIAgent decision-making and execution method based on long-term goals, characterized in that, Includes the following steps: S1. Obtain the current environmental state and map the current environmental state into a state feature vector through a preset state feature mapping function; S2. Structure the preset long-term goals into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries; S3. Based on the set of hard constraints, the set of trend constraints, and the set of deviable boundaries, calculate the target tension value corresponding to the state feature vector, thereby constructing the target tension field; S4. Generate a set of candidate actions based on the current environment state; S5. Perform state prediction for each candidate action in the candidate action set to obtain the corresponding predicted state, and map the predicted state to a predicted state feature vector. S6. Based on the predicted state feature vector, calculate the target tension change corresponding to each candidate action, and construct the candidate action evaluation value in combination with the risk assessment results. S7. Under the condition of satisfying the set of hard constraints, determine the target action from the set of candidate actions according to the evaluation value of the candidate actions; S8. Based on the relationship between the state feature vector and the set of deviable boundaries, adjust the execution intensity of the target action and execute the adjusted target action; S9. Perform consistency detection on the target tension values ​​corresponding to multiple consecutive time points. When the detection results meet the preset conditions, execute the target consistency reconstruction step.

2. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S2, the step of structuring the preset long-term goal into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries includes: The long-term goal is broken down into multiple target constraints; Based on the target constraint terms, the corresponding hard constraint conditions, trend constraint conditions, and deviation boundary conditions are determined respectively. The hard constraints are represented as a set of hard constraint functions, the trend constraints are represented as a set of trend constraint functions, and the deviable boundary conditions are represented as a set of deviable boundary functions. Furthermore, the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions all use the state feature vector as the input variable.

3. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S3, the step of calculating the target tension value corresponding to the state feature vector, thereby constructing the target tension field, includes: Obtain the function output results of the state feature vector in the set of hard constraint functions, the set of trend constraint functions, and the set of deviable boundary functions, respectively; The violation degree of each function output is calculated to obtain the corresponding hard constraint violation degree, trend violation degree, and boundary violation degree; The hard constraint tension, trend tension, and boundary tension are calculated based on the hard constraint violation degree, trend violation degree, and boundary violation degree, respectively. The hard constraint tension, trend tension, and boundary tension are then combined to obtain the target tension value.

4. The AIAgent decision-making and execution method based on long-term goals according to claim 3, characterized in that, The steps for calculating the violation degree of each function's output include: Determine if the output of the constraint function is greater than zero; When the output of the constraint function is less than or equal to zero, the corresponding violation degree is set to zero; When the output of the constraint function is greater than zero, the corresponding violation degree is determined based on the magnitude of the output value.

5. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S5, the step of predicting the state of each candidate action in the candidate action set includes: Use the current environment state and candidate actions as input parameters; Based on the input parameters, a one-step state prediction model is invoked to generate the predicted state; The predicted state is then converted into a predicted state feature vector using the state feature mapping function.

6. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S6, the step of constructing candidate action evaluation values ​​based on the risk assessment results includes: Calculate the target tension value corresponding to the predicted state feature vector and the target tension value corresponding to the current state feature vector, respectively. The change in target tension is determined based on two target tension values; Obtain the risk assessment results corresponding to the candidate actions; The candidate action evaluation value is obtained by combining the target tension change with the risk assessment result.

7. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S7, the step of determining the target action from the candidate action set based on the candidate action evaluation value includes: Based on the predicted state corresponding to the candidate action, determine whether each candidate action satisfies the set of hard constraints. The candidate actions that satisfy the set of hard constraints are taken as the set of optional actions; Then, the candidate action with the smallest evaluation value is selected from the set of optional actions as the target action.

8. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S8, the step of adjusting the execution intensity of the target action and executing the adjusted target action includes: Based on the state feature vector, calculate its function output in the set of deviable boundary functions; The corresponding execution intensity parameter is determined based on the function output. The target action is then performed based on the execution intensity parameter.

9. The AIAgent decision-making and execution method based on long-term goals according to claim 1, characterized in that, In step S9, the step of performing the target consistency reconstruction step when the detection result meets the preset conditions includes: Obtain the target tension values ​​corresponding to multiple consecutive time points; Construct a tension variation sequence based on the target tension value; Determine whether the preset reconstruction conditions are met based on the tension change sequence; When the preset reconstruction conditions are met, the weight parameters corresponding to the hard constraint set, trend constraint set, and deviation boundary set are updated, or a new candidate action set is generated.

10. An AIAgent decision-making and execution system based on long-term goals, characterized in that, An AIAgent decision-making execution method based on long-term goals, applicable to any one of claims 1-9, comprises: The state acquisition and feature mapping module is used to acquire the environment state and generate state feature vectors; The long-term goal structuring module is used to structure long-term goals into a set of hard constraints, a set of trend constraints, and a set of deviable boundaries. The target tension field construction module is used to construct the target tension field based on the set of hard constraints, the set of trend constraints, and the set of deviable boundaries. The candidate action generation and prediction module is used to generate a set of candidate actions and obtain the corresponding prediction state; The action evaluation and selection module is used to calculate the evaluation function of candidate actions and select the target action under the premise of satisfying the set of hard constraints. The execution adjustment module is used to adjust the execution intensity of the target action based on the determination result of the set of possible deviations from the boundary. The target consistency detection and reconstruction module is used to detect the consistency of the target tension state and trigger the target consistency reconstruction process when the conditions are met.

Images