Robot agent closed-loop skill scheduling method and system

Abstract

The application provides a robot intelligent agent closed-loop skill scheduling method and system, comprising: receiving a target task request of a user; decomposing the target task into multiple sub-target tasks; for each sub-target task, preliminarily screening a candidate skill set from a skill registry; performing basic qualification verification on each skill in the candidate skill set to meet preset requirements; only performing feasibility scoring and sorting on the candidate skills whose qualifications meet the preset requirements, and taking the best one as the target skill of the sub-target task; generating a restricted structured plan based on the sub-target task and the corresponding target skill; performing legality verification on the generated restricted structured plan; selecting a corresponding structured execution backend according to the characteristics of the sub-target task, instantiating the verified restricted structured plan into a corresponding execution graph; converting the execution graph into a standardized skill calling instruction, and driving the robot to execute the corresponding skill after bottom layer control conversion.

Description

Technical Field

[0001] This invention relates to the field of robot task planning technology, specifically to a closed-loop skill scheduling method and system for robot agents, and more specifically to a restricted planning, state gating, and behavior tree-based skill scheduling method and system for robot agents with closed loops. Background Technology

[0002] With the development of embodied intelligent robots, mobile manipulation robots, and large language model technology, robot systems are gradually evolving from single perception or single control modules into closed-loop intelligent agents encompassing "model-tool-memory-execution feedback." Users can use natural language or voice commands to instruct robots to complete long-range tasks such as grasping, placing, organizing, opening and closing, assembling, and navigating.

[0003] Existing natural language-controlled robot solutions typically select skills or generate task chains directly from a large language model. While this improves task generalization capabilities, it is prone to problems such as incorrect skill names, missing parameters, invalid states, unexecutable actions, and model illusions. For robotic arms and mobile chassis, incorrect skill parameters may lead to trajectory out-of-bounds errors, gripper malfunctions, increased collision risks, or controller alarms.

[0004] Existing behavior tree solutions can express task execution order, branch selection, retry, and recovery logic, but they typically rely on manual configuration of XML, JSON, or YAML files. While these solutions offer strong determinism, they lack automatic plan generation, dynamic parameter binding, and scene graph-based object slot completion capabilities when facing open-ended natural language tasks.

[0005] Existing finite state machine schemes are suitable for expressing safe states and event transitions, but the number of states can easily expand when the task size increases and the branching and looping structures become more complex. At the same time, traditional finite state machines lack a unified bridging mechanism between LLM generation plans, behavior tree recovery structures, skill registry entries, and controller feedback.

[0006] Existing LLM tools or multi-focused high-level task planning for robotic intelligent bodies do not fully incorporate robotic arm controller feedback, trajectory errors, gripper current, six-dimensional force, visual relocalization, collision risks, target loss, resource lock conflicts, and failure recovery strategies as part of closed-loop scheduling. Therefore, the system may encounter problems such as "the plan is correct but execution fails and cannot be recovered from," "LLM bypasses safety gating and directly issues dangerous actions," and "the behavior tree execution status cannot be fed back to the LLM for replanning."

[0007] Therefore, a closed-loop skill scheduling method for robot agents is needed, which restricts the open task understanding capability of VLM to a verifiable robot control protocol. The Agent is responsible for plan verification and parameter binding, the FSM is responsible for safety gating, the BT and SkillRunner are responsible for execution and recovery, and the controller and sensor feedback drive the recovery subtree, restricted replanning, and long-term memory updates. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide a closed-loop skill scheduling method and system for robot intelligent agents.

[0009] A closed-loop skill scheduling method for a robot agent according to the present invention includes: Step S1: Receive the user's target task request, and at the same time obtain the robot's own state, environment state and task state; and integrate the robot's own state, environment state, task state, historical execution experience, skill registry and safety rules into a unified intelligent agent context; Step S2: Based on the VLM meta controller, decompose the target task into multiple sub-target tasks, identify the constraints, success criteria and corresponding risk points of each sub-target task; for each sub-target task, preliminarily screen the candidate skill set from the skill registry. Step S3: Based on the robot's own state and the environment state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements; only perform feasibility scoring and sorting on the candidate skills whose qualifications meet the preset requirements, and take the one with the best score as the target skill for the sub-target task; when the qualified candidate skill set for a certain sub-target task is empty, abandon the current execution of the current sub-target task and trigger restricted replanning. Step S4: Based on the sub-target task and its corresponding target skill, scene graph query results, state summary, and preset template constraints, a constrained structured plan is generated through the VLM meta controller; wherein, the state summary includes: robot's own state, environment state, and task state summary; the scene graph query results are the task object-task object attribute-task object spatial relationships obtained by performing a structured query on the current scene graph for the sub-target task; Step S5: Perform a validity check on the generated restricted structured plan to ensure it meets preset requirements; Step S6: Select the corresponding structured execution backend based on the characteristics of the sub-target task, and instantiate the verified restricted structured plan into the corresponding execution graph; Step S7: Convert the execution diagram into standardized skill call instructions through a unified skill output, and drive the robot to execute the corresponding skill after conversion by the underlying control. Step S8: Collect the execution feedback from the controller and sensors in real time, convert it into structured execution results and error codes, write it to the shared blackboard, and update the unified agent context synchronously; Step S9: Determine the task execution status based on the execution feedback. If the execution is normal, proceed with the task until completion. If an execution exception or security event occurs, prioritize the execution of the local recovery strategy. If local recovery is ineffective, trigger VLM restricted replanning and update the execution graph.

[0010] Preferably, step S2, which decomposes the target task into multiple sub-target tasks based on the VLM meta-controller, includes: decomposing the target task at a granularity where each sub-target task can be completed by a closed loop of a single skill or a finite combination of atomic skills in the skill registry, and the sub-target task has a determinable success criterion, to obtain multiple sub-tasks.

[0011] Preferably, step S3 includes: Step S3.1: Based on the robot's own state and the environmental state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements, including: verifying whether the preconditions, parameter allowable domain satisfiability, tool constraints, workspace constraints, safety restrictions and sensor dependencies corresponding to each candidate skill are met; Step S3.2: Only qualified candidate skills are ranked by feasibility score, and the one with the highest score is selected as the target skill for the sub-target task; when the qualified candidate skill set for a sub-target task is empty, the current execution of the current sub-target task is abandoned and a restricted replanning is triggered. The skill feasibility score includes:

[0012] in, Represents the sub-target task g and the unified agent context. and state summary Feasibility rating for skill k; The following parameters represent the semantic matching degree between skills and tasks: Pre represents the degree of satisfaction of preconditions; Reach represents the reachability of the robotic arm or chassis; Risk represents the risk of collision, safety, or illegal state; Cost represents the expected execution cost; and Hist represents historical success priors. to The weighting coefficients; the state summary It includes a unified state summary consisting of the robot's own state, the environment's state, and the task's state.

[0013] Preferably, step S5 includes: Perform validity checks on the generated restricted structured plan, including:

[0014] in, Indicate whether the plan π is legal; K represents the skill invoked in the i-th planned step; K represents the set of registered skills. This represents the i-th skill parameter; Indicates the allowed fields for the corresponding parameters; Indicates the state Next event The corresponding state transition is valid; Indicates an indicator function; Let i be the total number of SkillCall calls, and i be the step number, i=1,2,…,N; When V(π)=0, the current restricted structured plan is determined to be invalid. The current restricted structured plan is then discarded and restricted replanning is triggered. A new candidate plan is generated and the plan's legality is checked again. If a legal plan cannot be obtained within the preset number of replanning attempts, the execution of the current sub-target task is abandoned and reported to the upper layer for task decomposition. The legality of state transitions is represented as:

[0015] in, and These represent the current security state and the next security state, respectively. Indicates the current event; Represents a predefined set of legal and safe migrations; State bus consistency include:

[0016] in, The result of the state consistency determination at time t is represented; M represents the set of modules participating in state synchronization. This indicates the current version number of module m; This indicates the reference version used in this round of planning; Indicates the maximum allowed version deviation; if If the consistency requirement is not met, new SkillCalls will be prohibited from being issued.

[0017] Preferably, step S6 includes: Instantiate the validated restricted structured plan into a BT, FSM, or SkillRunner execution graph; among them, FSM is suitable for formal conditional branching and state transitions; BT is suitable for concurrent monitoring, reactive scheduling, and recovery subtrees; SkillRunner is suitable for sequential long-chain tasks, row-level retries, and block-level rollbacks; When the execution graph is BT, construct nodes including sequence nodes, selection nodes, retry nodes, timeout nodes, condition nodes, action nodes, parallel monitoring nodes, and recovery nodes / abnormal recovery nodes; When the execution graph is SkillRunner, the execution state completes state_completion row expansion, row-level retries, and BlockTrie block-level rollbacks. When the execution graph is an FSM, legal transitions are performed according to the set of states and the set of events.

[0018] Preferably, step S7 includes: SkillFactory / SkillExecutor serving as a unified skill output, converting SkillCalls generated by FSM, BT, or SkillRunner into specific robot skill calls; wherein, SkillFactory is responsible for determining the skill number... The specific skill instance is constructed based on the skill call parameters bound to the query results of the scene graph; the SkillExecutor is responsible for scheduling the execution of the current skill instance and collecting the execution results.

[0019] Preferably, step S8 further includes: Receive execution feedback from the controller and sensors, including trajectory error, collision risk, gripper current, six-dimensional force, contact normal, object state change, visual relocation result, and controller alarm; convert the feedback into skill execution result SkillResult and structured error code, and write them to the Blackboard; The SkillResult includes: a success flag, error code, telemetry data, object state increment, and recovery strategy suggestion. The Blackboard is used to record skill execution status, error codes, scene changes, node status, and execution progress, and is used to update context information; The behavior tree node status score is calculated based on controller feedback and sensor feedback. This score is used to quantify the execution health of each behavior tree node and serves as the basis for the behavior tree to return a success, failure, or running status, as well as to trigger node retry, subtree recovery, or safety interruption. The behavior tree node state score is represented as follows:

[0020] in, This represents the execution score of behavior tree node n at time t; Indicates controller feedback; Indicates sensory feedback; Indicates the degree of satisfaction of the postcondition; This indicates a penalty for exceeding the time limit; Indicates a penalty for a dangerous situation; to These are the weighting coefficients; The blackboard summary is represented as follows:

[0021] in, This represents the compressed summary of the blackboard within the time window. For compression operation, Assign skill numbers, This serves as a marker for successful skill execution. For structured error codes, For incremental changes in the scene, The time sequence of skill execution within the time window [tT,t].

[0022] Preferably, step S9 includes: The priority of recovery actions is represented as follows:

[0023] in, The priority of the recovery action r is indicated; Recoverable(err) indicates the degree to which the current error is recoverable; SuccessHist(r) indicates the historical success rate of the recovery action; Delay(r) indicates the recovery time; Risk(r) indicates the additional risk introduced by the recovery action. to These are the weighting coefficients; When partial recovery is insufficient to resolve the issue, the decision to trigger restricted replanning is based on the cost of replanning.

[0024] in, This indicates the cost score for triggering a replanning process; Indicates the depth in the task tree where the failure occurred; This indicates uncertainty in the state; Indicates the magnitude of environmental change; This indicates the feasibility of partial restoration; to These are the weighting coefficients.

[0025] Preferably, the method further includes: The deployment trajectory quality score is expressed as follows:

[0026] Where Q(τ) represents the deployment trajectory quality score; SuccessRate(τ) represents the success rate of subtasks in the trajectory; Interrupt(τ) represents the number of multimodal interruptions; and LatencyGap(τ) represents the deviation of the actual time consumption from the end-to-end SLA budget. When deploying trajectory quality scoring When a preset threshold is met, a structured summary of the current trajectory is written into the historical execution experience. Trajectories that do not meet the threshold are not written to ensure the quality of the historical prior.

[0027] A closed-loop skill scheduling system for a robot agent according to the present invention includes: Module M1: Receives the user's target task request, and simultaneously obtains the robot's own state, environmental state, and task state; and integrates the robot's own state, environmental state, task state, historical execution experience, skill registry, and safety rules into a unified intelligent agent context; Module M2: Based on the VLM meta controller, the target task is decomposed into multiple sub-target tasks, and the constraints, success criteria and corresponding risk points of each sub-target task are identified; for each sub-target task, a set of candidate skills is initially screened from the skill registry. Module M3: Based on the robot's own state and the environment state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements; only perform feasibility scoring and ranking on the candidate skills whose qualifications meet the preset requirements, and select the one with the best score as the target skill for the sub-target task; when the qualified candidate skill set for a certain sub-target task is empty, abandon the current execution of the current sub-target task and trigger restricted replanning; Module M4: Based on the sub-target task and its corresponding target skill, scene graph query results, state summary, and preset template constraints, a restricted structured plan is generated through the VLM meta controller; wherein, the state summary includes: robot's own state, environment state, and task state summary; the scene graph query results are the task object-task object attribute-task object spatial relationships obtained by performing a structured query on the current scene graph for the sub-target task; Module M5: Performs a validity check on the generated restricted structured plan to ensure it meets preset requirements; Module M6: Selects the corresponding structured execution backend based on the characteristics of the sub-target task, and instantiates the validated restricted structured plan into the corresponding execution graph; Module M7: Converts the execution diagram into standardized skill call instructions through a unified skill output, and drives the robot to execute the corresponding skill after being converted by the underlying control. Module M8: Real-time acquisition of execution feedback from controllers and sensors, conversion into structured execution results and error codes, and writing to the shared blackboard, synchronously updating the unified agent context; Module M9: Determines the task execution status based on execution feedback. If execution is normal, the task is advanced until completion. If an execution exception or security event occurs, the local recovery strategy is executed first. If local recovery is ineffective, the VLM restricted replanning is triggered and the execution graph is updated.

[0028] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention can stably convert natural language tasks into constrained robot skill execution links. The VLM only outputs constrained structured plans or patches, and the Agent is responsible for plan verification, skill binding and parameter completion, thereby avoiding the model from directly generating uncontrollable low-level actions. 2. In this invention, FSM performs safety state gating on the planned segments, BT and SkillRunner only execute skills in legal states, and controller feedback and multimodal interrupt signals can trigger EARLY_STOP, pause, resume or safe shutdown, which can improve the safety of robotic arm task execution; 3. Behavior tree recovery subtree, SkillRunner row-level retries, BlockTrie block-level rollback, and exception code to recovery strategy matrix can reduce the number of whole task restarts, improve the recoverability of long chain tasks such as pick-place-navigate, and improve the success rate of long sequence tasks. 4. This invention can improve the accuracy of skill parameter binding. The system can automatically complete skill parameters based on scene diagrams, maps, target objects, container nodes, grabbing components, and historical success rates, reducing the risk of errors caused by LLM directly generating numerical parameters; 5. This invention can form a true Agent closed loop. Execution results, error codes, scene_diff, and controller feedback are written to the blackboard and compressed into the next round of LLM replanning context, making "perception-planning-execution-feedback-replanning" an explicit closed loop; 6. This invention can take into account the advantages of different structured execution backends. FSM is suitable for if / while formal branches, BT is suitable for concurrent listening and reactive recovery, SkillRunner is suitable for sequential long chains and block-level rollback, and the agent kernel can deterministically distribute or transfer tasks according to the characteristics of subtasks. 7. This invention can support a closed-loop lifecycle. Deployment trajectory, success rate, failure reasons and effective recovery path can be fed back into the training pool by quality gate for subsequent skill ranking, recovery subtree distillation, strategy learning and redeployment. 8. This invention can adapt to multiple actuators and cross-entity deployment. Through unified SkillFactoryProxy, resource locks and cross-entity action mapping, the same DSL or structured plan can be deployed to different entity combinations such as single-arm, dual-arm, mobile chassis plus robotic arm. Attached Figure Description

[0029] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is a schematic diagram of the overall architecture of a robot intelligent agent with the Agent at its core, representing a closed-loop execution process.

[0030] Figure 2 This is a diagram illustrating the Agent-Executor Commander-Staff collaboration and lifecycle closed loop. Detailed Implementation

[0031] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0032] Example 1 According to the present invention, a closed-loop skill scheduling method and system for robot intelligent agents is provided, which uses Robot Agent as the top-level intelligent agent core, receives natural language tasks, multimodal observations, robot states, scene graphs, skill registry entries and security policies, and converts tasks into verifiable, executable, interruptible and resumable robot skill execution graphs.

[0033] The closed-loop skill scheduling method for the robot agent includes: Step S1: Receive the user's target task request, and at the same time obtain the robot's own state, environment state and task state; and integrate the robot's own state, environment state, task state, historical execution experience, skill registry and safety rules into a unified intelligent agent context; Step S2: Based on the VLM meta controller, decompose the target task into multiple sub-target tasks, identify the constraints, success criteria and corresponding risk points of each sub-target task; for each sub-target task, preliminarily screen the candidate skill set from the skill registry. Specifically, step S2, which decomposes the target task into multiple sub-target tasks based on the VLM meta controller, includes: decomposing the target task at a granularity where each sub-target task can be completed by a closed loop of a single skill or a finite combination of atomic skills in the skill registry, and the sub-target task has a determinable success criterion, to obtain multiple sub-tasks.

[0034] Step S3: Based on the robot's own state and the environment state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements; only perform feasibility scoring and sorting on the candidate skills whose qualifications meet the preset requirements, and take the one with the best score as the target skill for the sub-target task; when the qualified candidate skill set for a certain sub-target task is empty, abandon the current execution of the current sub-target task and trigger restricted replanning. Specifically, step S3 includes: Step S3.1: Based on the robot's own state and the environmental state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements, including: verifying whether the preconditions, parameter allowable domain satisfiability, tool constraints, workspace constraints, safety restrictions and sensor dependencies corresponding to each candidate skill are met; Step S3.2: Only qualified candidate skills are ranked by feasibility score, and the one with the highest score is selected as the target skill for the sub-target task; when the qualified candidate skill set for a sub-target task is empty, the current execution of the current sub-target task is abandoned and a restricted replanning is triggered. The skill feasibility score includes:

[0035] in, Represents the sub-target task g and the unified agent context. and state summary Feasibility rating for skill k; The following parameters represent the semantic matching degree between skills and tasks: Pre represents the degree of satisfaction of preconditions; Reach represents the reachability of the robotic arm or chassis; Risk represents the risk of collision, safety, or illegal state; Cost represents the expected execution cost; and Hist represents historical success priors. to The weighting coefficients; the state summary It includes a unified state summary consisting of the robot's own state, the environment's state, and the task's state.

[0036] Step S4: Based on the sub-target task and its corresponding target skill, scene graph query results, state summary, and preset template constraints, a constrained structured plan is generated through the VLM meta controller; wherein, the state summary includes: robot's own state, environment state, and task state summary; the scene graph query results are the task object-task object attribute-task object spatial relationships obtained by performing a structured query on the current scene graph for the sub-target task; Step S5: Perform a validity check on the generated restricted structured plan to ensure it meets preset requirements; Specifically, step S5 includes: Perform validity checks on the generated restricted structured plan, including:

[0037] in, Indicate whether the plan π is legal; K represents the skill invoked in the i-th planned step; K represents the set of registered skills. This represents the i-th skill parameter; Indicates the allowed fields for the corresponding parameters; Indicates the state Next event The corresponding state transition is valid; Indicates an indicator function; Let i be the total number of SkillCall calls, and i be the step number, i=1,2,…,N; When V(π)=0, the current restricted structured plan is determined to be invalid. The current restricted structured plan is then discarded and restricted replanning is triggered. A new candidate plan is generated and the plan's legality is checked again. If a legal plan cannot be obtained within the preset number of replanning attempts, the execution of the current sub-target task is abandoned and reported to the upper layer for task decomposition. The legality of state transitions is represented as:

[0038] in, and These represent the current security state and the next security state, respectively. Indicates the current event; Represents a predefined set of legal and safe migrations; State bus consistency include:

[0039] in, The result of the state consistency determination at time t is represented; M represents the set of modules participating in state synchronization. This indicates the current version number of module m; This indicates the reference version used in this round of planning; Indicates the maximum allowed version deviation; if If the consistency requirement is not met, new SkillCalls will be prohibited from being issued.

[0040] Step S6: Select the corresponding structured execution backend based on the characteristics of the sub-target task, and instantiate the verified restricted structured plan into the corresponding execution graph; Specifically, step S6 includes: Instantiate the validated restricted structured plan into a BT, FSM, or SkillRunner execution graph; among them, FSM is suitable for formal conditional branching and state transitions; BT is suitable for concurrent monitoring, reactive scheduling, and recovery subtrees; SkillRunner is suitable for sequential long-chain tasks, row-level retries, and block-level rollbacks; When the execution graph is BT, construct nodes including sequence nodes, selection nodes, retry nodes, timeout nodes, condition nodes, action nodes, parallel monitoring nodes, and recovery nodes / abnormal recovery nodes; When the execution graph is SkillRunner, the execution state completes state_completion row expansion, row-level retries, and BlockTrie block-level rollbacks. When the execution graph is an FSM, legal transitions are performed according to the set of states and the set of events.

[0041] Step S7: Convert the execution diagram into standardized skill call instructions through a unified skill output, and drive the robot to execute the corresponding skill after conversion by the underlying control. Specifically, step S7 includes: SkillFactory / SkillExecutor, acting as a unified skill output, converts SkillCalls generated by FSM, BT, or SkillRunner into specific robot skill calls; wherein, SkillFactory is responsible for determining the skill number... The specific skill instance is constructed based on the skill call parameters bound to the query results of the scene graph; the SkillExecutor is responsible for scheduling the execution of the current skill instance and collecting the execution results.

[0042] Step S8: Collect the execution feedback from the controller and sensors in real time, convert it into structured execution results and error codes, write it to the shared blackboard, and update the unified agent context synchronously; Specifically, step S8 further includes: Receive execution feedback from the controller and sensors, including trajectory error, collision risk, gripper current, six-dimensional force, contact normal, object state change, visual relocation result, and controller alarm; convert the feedback into skill execution result SkillResult and structured error code, and write them to the Blackboard; The SkillResult includes: a success flag, error code, telemetry data, object state increment, and recovery strategy suggestion. The Blackboard is used to record skill execution status, error codes, scene changes, node status, and execution progress, and is used to update context information; The behavior tree node status score is calculated based on controller feedback and sensor feedback. This score is used to quantify the execution health of each behavior tree node and serves as the basis for the behavior tree to return a success, failure, or running status, as well as to trigger node retry, subtree recovery, or safety interruption. The behavior tree node state score is represented as follows:

[0043] in, This represents the execution score of behavior tree node n at time t; Indicates controller feedback; Indicates sensory feedback; Indicates the degree of satisfaction of the postcondition; This indicates a penalty for exceeding the time limit; Indicates a penalty for a dangerous situation; to These are the weighting coefficients; The blackboard summary is represented as follows:

[0044] in, This represents the compressed summary of the blackboard within the time window. For compression operation, Assign skill numbers, This serves as a marker for successful skill execution. For structured error codes, For incremental changes in the scene, The time sequence of skill execution within the time window [tT,t].

[0045] Step S9: Determine the task execution status based on the execution feedback. If the execution is normal, proceed with the task until completion. If an execution exception or security event occurs, prioritize the execution of the local recovery strategy. If local recovery is ineffective, trigger VLM restricted replanning and update the execution graph.

[0046] Specifically, step S9 includes: The priority of recovery actions is represented as follows:

[0047] in, The priority of the recovery action r is indicated; Recoverable(err) indicates the degree to which the current error is recoverable; SuccessHist(r) indicates the historical success rate of the recovery action; Delay(r) indicates the recovery time; Risk(r) indicates the additional risk introduced by the recovery action. to These are the weighting coefficients; When partial recovery is insufficient to resolve the issue, the decision to trigger restricted replanning is based on the cost of replanning.

[0048] in, This indicates the cost score for triggering a replanning process; Indicates the depth in the task tree where the failure occurred; This indicates uncertainty in the state; Indicates the magnitude of environmental change; This indicates the feasibility of partial restoration; to These are the weighting coefficients.

[0049] Specifically, the method further includes: The deployment trajectory quality score is expressed as follows:

[0050] Where Q(τ) represents the deployment trajectory quality score; SuccessRate(τ) represents the success rate of subtasks in the trajectory; Interrupt(τ) represents the number of multimodal interruptions; and LatencyGap(τ) represents the deviation of the actual time consumption from the end-to-end SLA budget. When deploying trajectory quality scoring When a preset threshold is met, a structured summary of the current trajectory is written into the historical execution experience. Trajectories that do not meet the threshold are not written to ensure the quality of the historical prior.

[0051] The present invention also provides a closed-loop skill scheduling system for a robot intelligent agent. The closed-loop skill scheduling system for a robot intelligent agent can be implemented by executing the process steps of the closed-loop skill scheduling method for a robot intelligent agent. That is, those skilled in the art can understand the closed-loop skill scheduling method for a robot intelligent agent as a preferred embodiment of the closed-loop skill scheduling system for a robot intelligent agent.

[0052] Example 2 The robot intelligent agent closed-loop skill scheduling method provided by the present invention includes: unifying natural language tasks, multimodal observation, structured memory, skill registry, FSM safety gating, BT behavior tree execution, SkillRunner sequential execution, SkillFactory unified skill exit, and controller feedback into the same closed-loop runtime, enabling the robot to complete verifiable, interruptible, recoverable, and updatable long-range operation tasks.

[0053] The closed-loop skill scheduling method for the robot agent includes: Step 1: Receive task input and multimodal status; The Robot Agent receives natural language, voice, or task input from a higher-level system, and simultaneously reads vision, depth, force, joint status, end effector status, scene graph, map status, long-term memory, skill registry, and safety policies.

[0054] In this step, the basic runtime states are established, including: robot state, environment state, and task state; The robot state includes at least: joint angles, tool center point pose, gripper state, load weight, controller mode, and safety state. The environmental state includes at least: scene map version, set of visible objects, obstacle information, personnel safety zone, and map incremental update data; The task status includes at least: task number, target parameter slot, constraints, task progress, and task completion determination conditions.

[0055] Step 2: Construct a unified agent context; The Robot Agent constructs a unified agent context, which includes robot state, environment state, task state, historical execution summary, available tool summary, skill registry summary, and security policy summary.

[0056] The historical execution summary can be derived from the execution trajectory, which includes at least: timestamp, plan number, node number, feedback information, and execution result. This summary records historical success rates, reasons for failure, recovery paths, and scenario changes, providing a basis for subsequent plan generation and skill ranking.

[0057] Step 3: Understand the execution intent and decompose subtasks, filter candidate skills and check execution conditions; The VLM meta-controller is invoked for high-level intent understanding, decomposing the task into sub-goals, object slots, constraints, success criteria, and risk warnings. The VLM only generates high-level constrained planning information and does not directly generate joint angle, motor current, or servo cycle control quantities.

[0058] For each sub-target task, a set of candidate skills is initially obtained by screening from the skill registry. Each skill in the skill registry includes at least: skill number, parameter definition domain, preconditions, postconditions, timeout threshold, resource lock, risk level, sensor requirements, error code, and exception recovery entry. If a skill lacks key fields or its parameters do not meet the parameter domain constraints, it cannot proceed to the next execution stage. Specifically, based on the current robot state and environment state, each candidate skill is checked against its corresponding key fields, such as preconditions, parameter range, tool constraints, workspace constraints, safety restrictions, and sensor dependencies. If these are not met, the candidate skill will not proceed to the next execution stage. Candidate skills that pass the qualification verification form a qualified set. Feasibility scores are then calculated and ranked for the skills in the qualified set, and the skill with the highest score is selected as the target skill for the subtask. When the qualified set is empty, the current execution of the subtask is abandoned, triggering a restricted replanning process, and the sub-target sequence is re-decomposed or adjusted by the upper layer.

[0059] When generating candidate skills, a skill feasibility score is calculated based on task semantics, current state, and historical experience: (1) in, Represents the sub-target task g and the unified agent context. and state summary Feasibility rating for skill k; The following parameters represent the semantic matching degree between skills and tasks: Pre represents the degree of satisfaction of preconditions; Reach represents the reachability of the robotic arm or chassis; Risk represents the risk of collision, safety, or illegal state; Cost represents the expected execution cost; and Hist represents historical success priors. to The weighting coefficients; the state summary It includes a unified state summary consisting of the robot's own state, the environment's state, and the task's state.

[0060] Step 4: Generate a restricted structured plan (PlanCandidate) and perform validation; The task description, candidate skills, scene graph query results, state summary, and template constraints are input into the VLM meta-controller to generate a restricted structured plan, PlanCandidate. The restricted structured plan, PlanCandidate, includes at least: a plan number, a skill sequence, a parameter set, preconditions, postconditions, and fallback strategies. The task description is the sub-task description obtained from decomposition; the candidate skills refer to the target skills and their alternative qualified skills that have passed qualification verification and are ranked and selected according to feasibility scores; the state summary is a summary of the robot's state, environmental state, and task state. The VLM meta-controller uses the skill registry and template constraints as constraints on selectable skills and output formats, mapping sub-tasks to skill sequences composed of candidate skills and binding parameters, outputting a restricted structured plan, PlanCandidate, that satisfies a predefined schema. This generation process is further constrained by the Agent's verification in step 5, ensuring that its output is limited to a verifiable robot control protocol, rather than directly generating low-level control quantities such as joint angles and motor currents from common knowledge in the field.

[0061] The legality of the plan can be expressed as: (2) in, Indicate whether the plan π is legal; K represents the skill invoked in the i-th planned step; K represents the set of registered skills. This represents the i-th skill parameter; Indicates the allowed fields for the corresponding parameters; Indicates the state Next event The corresponding state transition is valid; Indicates an indicator function; Let i be the total number of SkillCall calls, and i be the step number, i=1,2,…,N; when When the value is 0, it indicates that the plan contains unregistered skills, skill parameters exceeding the allowed range, or invalid state transitions. In this case, the restricted structured plan is deemed invalid, triggering restricted regeneration, supplementary sensing, or an error return, and it cannot be executed. It should be noted that since the candidate skills have already undergone unique verification of their registration existence in the initial screening stage of step 3, and the restricted structured plan is assembled from the SkillCall sequence of verified skills, therefore… In The factors are guaranteed to be true by construction and are only retained as a formal expression of the formula's completeness; no repeated existence verification operations are actually performed. The actual determination is a joint determination of two factors: whether the actual call parameters after scene graph binding fall within the allowed domain Ω, and whether the FSM timing of the SkillCall sequence is valid; where here... The specific call parameters after being bound by the scene graph belong to different granularities than the "skill abstract parameter domain satisfiability" determined in step 3 qualification verification.

[0062] The Agent performs schema validation, parameter completion, scene graph parameter binding, state validity validation, permission validation, and dangerous action filtering on the restricted structured plan PlanCandidate. If the validation fails, it triggers restricted regeneration, supplementary awareness, or an error return. Schema validation → Scene graph parameter binding / completion → After binding, check if param∈Ω (unique time) → FSM sequence validity (plan level) → Permissions and Dangerous Action Filtering → State Bus Consistency The above are all static verifications of the predicted state during the planning phase. FSM safety gating, on the other hand, verifies the actual robot state during the execution phase. With actual events The gradual dynamic guarding, with different inputs (predicted state and measured state) and timings (planning period and execution period), respectively undertakes two independent lines of defense: "plan correctness" and "operational safety," and does not constitute duplicate verification.

[0063] The state bus consistency include: (3) in, The result of the state consistency determination at time t is represented; M represents the set of modules participating in state synchronization. This indicates the current version number of module m; This indicates the reference version used in this round of planning; This indicates the maximum allowed version deviation. If If the consistency requirement is not met, new SkillCalls will be prohibited from being issued.

[0064] Based on the current security state and the set of legal migrations, the FSM performs security gating on planned segments, controlling the system to legally migrate between standby, sensing, planning, pre-check, execution, verification, recovery, safe shutdown, and completion states.

[0065] The legality of state transitions can be represented as : (4) in, and These represent the current security state and the next security state, respectively. Indicates the current event; This represents a predefined set of legal and safe transitions. This formula ensures that the robot can only transition between permitted safe states.

[0066] Step 5: Select the structured execution backend and generate the execution graph; The plan, validated by the Agent, can be instantiated as a BT, FSM, or SkillRunner execution graph. The appropriate structured execution backend is selected based on the characteristics of the subtasks: FSM is suitable for formal conditional branching and state transitions; BT is suitable for concurrent monitoring, reactive scheduling, and subtree recovery; SkillRunner is suitable for sequential long-chain tasks, row-level retries, and block-level rollbacks. Here, "plan" refers to the constrained structured plan PlanCandidate that has passed Agent validation.

[0067] Actuator selection can be expressed as: (5) in, Indicates the selected structured execution backend; Indicates the complexity of the control structure; This indicates a concurrent listening requirement; Indicates the length of the sequential skill chain; This indicates a risk of contact, force control, or human-machine proximity. Represents the scheduling decision function; When the execution graph is BT, construct nodes such as sequence nodes, selection nodes, retry nodes, timeout nodes, condition nodes, action nodes, parallel monitoring nodes, and recovery nodes / abnormal recovery nodes; When the execution graph is SkillRunner, the execution state completes state_completion row expansion, row-level retries, and BlockTrie block-level rollbacks. When the execution graph is FSM, legal transitions are performed according to the state set and event set, that is, the FSM security gating and state transition legality determination are reused.

[0068] Step 6: Execute robot skills via SkillFactory; SkillFactory / SkillExecutor serves as a unified skill exporter, converting SkillCalls generated by FSM, BT, or SkillRunner into specific robot skill calls. Each skill call includes at least: skill number, input parameters, timeout in milliseconds, resource lock, and risk level. SkillFactory and SkillExecutor are two collaborative components of the same unified skill exporter: SkillFactory is responsible for... The parameters construct a specific skill instance, and SkillExecutor is responsible for scheduling the execution of the skill instance and collecting the execution results; together, they convert the upper-level skill call SkillCall into a specific robot skill execution.

[0069] SkillExecutor can invoke skills such as navigation, positioning, grasping, suction, placement, opening and closing, re-inspection, and safe retreat, and send target pose, trajectory, speed, force control parameters, or other control parameters to the robotic arm, gripper, suction cup, or mobile chassis through PrimitiveFactory / controller.

[0070] In this invention, VLM performs low-frequency task understanding and plan generation, FSM / BT performs mid-frequency state gating and tick scheduling, and high-frequency servo control is performed by the robotic arm controller or motion control module.

[0071] Step 7: Receive feedback from the controller and update the blackboard; The system receives execution feedback from the controller and sensors, including trajectory error, collision risk, gripper current, six-dimensional force, contact normal, object state changes, visual relocation results, and controller alarms. This feedback is converted into SkillResult and structured error codes and written to the Blackboard.

[0072] The SkillResult, the execution result, includes at least: a success flag, an error code, telemetry data, object state increments, and a recovery strategy suggestion. The Blackboard records the skill execution status, error codes, scene changes, node status, and execution progress, and can serve as the context for the next round of constrained replanning in VLM.

[0073] The state score of a behavior tree node can be represented as: (6) in, This represents the execution score of behavior tree node n at time t; Indicates controller feedback; Indicates sensory feedback; Indicates the degree of satisfaction of the postcondition; This indicates a penalty for exceeding the time limit; Indicates a penalty for a dangerous situation; to These are weighting coefficients. Behavior tree node state scores. Based on received controller feedback With perceptual feedback The calculation aims to quantify the execution health of each behavior tree node, serving as the basis for the behavior tree to return to the SUCCESS, FAILURE, or RUNNING state, as well as triggering node retry, subtree recovery, or safe interruption in step S11.

[0074] The blackboard summary can be represented as: (7) in, This represents the compressed summary of the blackboard within the time window. For compression operation, Assign skill numbers, This serves as a marker for successful skill execution. For structured error codes, For incremental changes in the scene, The time sequence for skill execution within the time window [tT,t]; This summary can serve as the context for the next round of VLM replanning, achieving a closed loop of "execution results - blackboard - next round of VLM plan".

[0075] Step 8: Trigger a security interruption, recovery strategy, or restricted replanning; When an action fails, the target is lost, the grab slips, the collision risk increases, parameters exceed limits, personnel approach, the object moves, or the controller alarms, the local recovery subtree is executed first. This includes re-sensing, re-localization, re-grabbing, retreating to a safe position, switching skills, or degrading execution. If failures occur consecutively, the VLM is invoked to generate restricted recovery suggestions, which are then inserted into the execution graph after being verified by the Agent.

[0076] A multimodal interrupt trigger can be represented as: (8) in, This indicates a safety interrupt trigger signal; Indicates an incident involving the approach of people; Indicates an object movement event; Indicates an abnormal external force event; Indicates a grab slip event; This indicates the appearance of a new object. When any signal is set, the robot enters a paused or stopped state and injects an EARLY_STOP early termination signal into the PrimitiveFactory. The multimodal interrupt trigger is used to integrate multimodal safety events such as personnel approach, object movement, external force anomalies, grasping slippage, and the appearance of a new object in real time during execution. Once any event is set, a safety interruption is triggered, allowing the robot to promptly enter a paused or safe stop state to ensure human-robot safety and provide triggering conditions for subsequent recovery strategies or restricted replanning.

[0077] The priority of recovery actions can be expressed as: (9) in, The priority of the recovery action r is indicated; Recoverable(err) indicates the degree to which the current error is recoverable; SuccessHist(r) indicates the historical success rate of the recovery action; Delay(r) indicates the recovery time; Risk(r) indicates the additional risk introduced by the recovery action. to These are the weighting coefficients.

[0078] When partial recovery is insufficient to resolve the issue, the cost of replanning can be used to determine whether restricted replanning should be triggered. (10) in, This indicates the cost score for triggering a replanning process; Indicates the depth in the task tree where the failure occurred; This indicates uncertainty in the state; Indicates the magnitude of environmental change; This indicates the feasibility of partial restoration; to These are the weighting coefficients.

[0079] Step 9: Write into memory, update experience, and form a closed life cycle; The task execution trajectory, success rate, failure reasons, effective recovery subtree, parameter correction results, and scene graph changes are written into short-term memory and long-term memory for subsequent skill sorting, plan constraints, context compression, and experience updates.

[0080] The historical success rate of a skill can be expressed as : (11) in, This represents the historical success rate of skill k at time t; This indicates the result of this execution; This is for updating coefficients. This formula is used to incorporate deployment and execution experience into subsequent skill ranking and candidate skill selection. (Skill historical success rate) This is the source of the Hist term, which represents the historical success priors, in the skill feasibility scoring formula (Formula 1): The prior statistical and online update form, after normalization, is used as Hist to backfill into formula (1) for scoring, i.e., Hist = Normalize( These two are different manifestations of the same quantity.

[0081] The deployment trajectory quality score can be expressed as: (12) Where Q(τ) represents the deployment trajectory quality score; SuccessRate(τ) represents the success rate of subtasks in the trajectory; Interrupt(τ) represents the number of multimodal interruptions; and LatencyGap(τ) represents the deviation of actual time from the end-to-end SLA budget. Trajectories that meet the quality threshold can be re-trained in the pool for policy learning, recovery subtree optimization, and redeployment. Specifically, when the deployment trajectory quality score... When a preset threshold is met, the structured summary of the trajectory is written into the historical execution summary (ExecutionTrace). Trajectories that do not meet the threshold are not written to ensure the quality of historical priors. The written ExecutionTrace is used for two purposes: firstly, to construct the unified agent context in steps S1 and S2, and secondly, to update the historical success rate of the skill. This forms a closed loop with the aforementioned scoring prior. Furthermore, in this invention, three types of events—an empty qualified set, an invalid plan V(π)=0, and runtime execution failure—are uniformly managed by the same restricted replanning / recovery controller and share the same global retry budget: local recovery is prioritized, followed by restricted replanning at the sub-objective level; if still unsolvable, the current sub-objective is abandoned and reported, thus ensuring that the number of retries in the overall process is bounded and the closed loop converges.

[0082] like Figure 2 As shown, this invention models the robot agent execution framework as a two-tiered "commander-staff" system. The commander is the VLM meta-controller, responsible for understanding the task, maintaining memory, selecting actuators, and triggering restricted replanning. The three staff units include FSM, BT, and SkillRunner, which are suitable for formal branching, safe state gating, concurrent listening, behavior tree recovery, sequential long chains, and block-level rollback, respectively. The three units invoke the underlying robot skills through a unified SkillFactoryProxy and provide feedback to the agent core through blackboards and actuator state summaries.

[0083] Example 3 Example 3 is a preferred example of Example 1. The robot intelligent agent closed-loop skill scheduling system provided by the present invention, such as Figure 1As shown, the system includes: a task input module, a multimodal observation module, a VLM meta-controller, a three-layer structured memory module, an Agent, an FSM safety gating module, a BT behavior tree executor, a SkillRunner, a SkillFactory / SkillExecutor, a PrimitiveFactory / controller, and a RecoveryPolicy module. These modules are connected via a unified state bus, blackboard data structure, and skill invocation protocol, forming a closed loop from task understanding, structured planning, state gating, skill execution to feedback recovery. The VLM only allows the generation of restricted structured plans or behavior tree patches and does not directly output low-level control quantities such as joint angles, motor currents, and servo cycle commands. All low-level actions are converted from SkillCall to target poses, trajectories, or skill parameters executable by the controller via SkillFactory and PrimitiveFactory.

[0084] The task input module receives task requests from users in natural language, voice commands, or from a host system, and converts them into a raw task description. The raw task description may include the target object, target location, operation type, constraints, and completion requirements. This module connects upstream to the user or business system and downstream to the robot's intelligent agent core.

[0085] The multimodal observation module is used to read environmental and body state information during robot operation, including RGB-D images, depth information, force information, joint states, end effector states, scene graph states, and map states. This module outputs state data such as RobotState, WorldState, and TaskState, providing real-time data for task understanding, plan generation, state gating, and execution verification.

[0086] The VLM meta-controller is used to perform natural language understanding, multimodal semantic understanding, subtask decomposition, object slot extraction, constraint identification, and constrained plan generation. This module generates a PlanCandidate based on task input, state summary, skill registry, and structured memory. The VLM meta-controller does not directly output joint angle, motor current, or servo cycle control commands; instead, it outputs constrained structured plans or actuator patches under the constraints of the Agent.

[0087] The three-layer structured memory module is used to maintain role memory, task memory, and working memory. Role memory records the robot's identity, capability boundaries, and available tools; task memory records the sub-objectives, dependencies, constraints, and completion status of the current task; and working memory records recent execution trajectories, tool call results, error codes, and scene changes. This module forms a closed loop with the blackboard, execution trajectory, and VLM input context, used to compress long-range contexts and support subsequent replanning.

[0088] The Agent is used to perform structured validation and execution constraints on the PlanCandidate output by the VLM. This module performs at least schema validation, parameter completion, scene graph parameter binding, permission validation, state validity validation, and dangerous action filtering. Skill existence has already been uniquely verified during the initial screening of candidate skills, and this module does not perform the validation again. Validated plan fragments are converted into structured tasks executable by FSM, BT, or SkillRunner. Plans that fail validation are subject to restricted regeneration, supplementary awareness, or error return.

[0089] The FSM safety gating module is used to determine whether a state transition is legal based on the current safety status and event set, and to control the system to transition between states such as standby, sensing, planning, pre-checking, execution, verification, recovery, safe shutdown, and completion. This module receives plan segments, execution events, and sensor feedback, constrains the reachable states of the actuator, and triggers pause, recovery, or safe shutdown when a dangerous state or illegal transition occurs.

[0090] The BT behavior tree actuator is used to organize robot skill execution through nodes such as Sequence, Selector, Retry, Timeout, Condition, Action, Parallel Monitor, and Recovery. This module is suitable for handling concurrent monitoring, condition judgment, failure retries, and recovery subtree scheduling. During BT execution, node status, execution results, error codes, and progress are written to the blackboard, and the SUCCESS, FAILURE, or RUNNING status is dynamically returned based on sensor feedback.

[0091] SkillRunner is used to execute sequential skill chains in DSL format, suitable for handling long-sequence production paths and structured task scripts. This module supports state_completion line expansion, line-level retries, precondition verification, and BlockTrie block-level rollback. When a single skill fails, SkillRunner can perform line-level retries according to the configuration; when there are causal dependencies between multiple skills, the matched skill block can be rolled back and re-executed.

[0092] The SkillFactory / SkillExecutor serves as the unified skill output for FSM, BT, and SkillRunner, converting upper-level structured tasks into specific robot skill calls. Based on the skill_id, parameters, timeout, resource locks, and risk level in the SkillCall, this module invokes robot skills such as navigation, localization, grasping, picking up, placing, opening / closing, re-inspection, and safe retreat, and returns the execution results to the blackboard and the upper-level scheduling module.

[0093] The PrimitiveFactory / controller is used to drive the robotic arm, gripper, suction cup, or mobile chassis to perform underlying motions. This module receives target pose, trajectory, velocity, force control parameters, or other control parameters, and calls the robot controller, gripper controller, or chassis controller to execute specific actions. During execution, this module returns feedback information such as trajectory error, gripper current, six-dimensional force, collision risk, visual repositioning results, and controller alarms.

[0094] The RecoveryPolicy module determines the recovery method based on error codes, sensor feedback, and FSM events. These recovery methods include, but are not limited to, re-sensing, re-localization, re-capture, returning to a safe position, switching skills, requesting supplemental sensing, restricted replanning, and safe shutdown. This module connects with the BT recovery subtree, the SkillRunner retry mechanism, and the Agent restricted replanning mechanism to prevent a direct system restart after task failure.

[0095] Through the collaborative work of the above modules, this invention forms a closed-loop robot agent execution framework consisting of "task input—multimodal observation—constrained plan generation—structured verification—state gating—behavior tree / sequential execution—skill invocation—controller feedback—recovery and replanning—memory update".

[0096] Those skilled in the art will understand that, in addition to implementing the system, apparatus, and their modules provided by this invention in purely computer-readable program code, the same program can be implemented in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system, apparatus, and their modules provided by this invention can be considered a hardware component, and the modules included therein for implementing various programs can also be considered structures within the hardware component; alternatively, modules for implementing various functions can be considered both software programs implementing the method and structures within the hardware component.

[0097] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A closed-loop skill scheduling method for a robot intelligent agent, characterized in that, include: Step S1: Receive the user's target task request, and at the same time obtain the robot's own status, environmental status and task status; The robot's own state, environmental state, task state, historical execution experience, skill registry, and safety rules are all integrated into a unified intelligent agent context; Step S2: Based on the VLM meta controller, decompose the target task into multiple sub-target tasks, and identify the constraints, success criteria and corresponding risk points of each sub-target task; For each sub-target task, a set of candidate skills is initially obtained by screening from the skill registry. Step S3: Based on the robot's own state and the environment state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements; only perform feasibility scoring and ranking on the candidate skills whose qualifications meet the preset requirements, and take the one with the best score as the target skill for the sub-target task; If the qualified candidate skill set for a certain sub-target task is empty, then the current execution of the current sub-target task is abandoned and a restricted replanning is triggered. Step S4: Based on the sub-target task and its corresponding target skill, scene graph query results, state summary, and preset template constraints, a constrained structured plan is generated through the VLM meta controller; wherein, the state summary includes: robot's own state, environment state, and task state summary; the scene graph query results are the task object-task object attribute-task object spatial relationships obtained by performing a structured query on the current scene graph for the sub-target task; Step S5: Perform a validity check on the generated restricted structured plan to ensure it meets preset requirements; Step S6: Select the corresponding structured execution backend based on the characteristics of the sub-target task, and instantiate the verified restricted structured plan into the corresponding execution graph; Step S7: Convert the execution diagram into standardized skill call instructions through a unified skill output, and drive the robot to execute the corresponding skill after conversion by the underlying control. Step S8: Collect the execution feedback from the controller and sensors in real time, convert it into structured execution results and error codes, write it to the shared blackboard, and update the unified agent context synchronously; Step S9: Determine the task execution status based on the execution feedback. If the execution is normal, proceed with the task until completion. If an execution exception or security event occurs, prioritize the execution of the local recovery strategy. If local recovery is ineffective, trigger VLM restricted replanning and update the execution graph.

2. The closed-loop skill scheduling method for a robot intelligent agent according to claim 1, characterized in that, Step S2, which decomposes the target task into multiple sub-target tasks based on the VLM meta-controller, includes: decomposing the target task at a granularity where each sub-target task can be completed by a closed loop of a single skill or a finite combination of atomic skills in the skill registry, and the sub-target task has a determinable success criterion, to obtain multiple sub-tasks.

3. The closed-loop skill scheduling method for a robot agent according to claim 1, characterized in that, Step S3 includes: Step S3.1: Based on the robot's own state and the environmental state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements, including: verifying whether the preconditions, parameter allowable domain satisfiability, tool constraints, workspace constraints, safety restrictions and sensor dependencies corresponding to each candidate skill are met; Step S3.2: Only qualified candidate skills are ranked by feasibility score, and the one with the highest score is selected as the target skill for the sub-target task; when the qualified candidate skill set for a sub-target task is empty, the current execution of the current sub-target task is abandoned and a restricted replanning is triggered. The skill feasibility score includes: in, Represents the sub-target task g and the unified agent context. and state summary Feasibility rating for skill k; The following parameters represent the semantic matching degree between skills and tasks: Pre represents the degree of satisfaction of preconditions; Reach represents the reachability of the robotic arm or chassis; Risk represents the risk of collision, safety, or illegal state; Cost represents the expected execution cost; and Hist represents historical success priors. to The weighting coefficients; the state summary It includes a unified state summary consisting of the robot's own state, the environment's state, and the task's state.

4. The closed-loop skill scheduling method for a robot agent according to claim 1, characterized in that, Step S5 includes: Perform validity checks on the generated restricted structured plans, including: in, Indicate whether the plan π is legal; K represents the skill invoked in the i-th planned step; K represents the set of registered skills. This represents the i-th skill parameter; Indicates the allowed fields for the corresponding parameters; Indicates the state Next event The corresponding state transition is valid; Indicates an indicator function; Let i be the total number of SkillCall calls, and i be the step number, i=1,2,…,N; When V(π)=0, the current restricted structured plan is determined to be invalid. The current restricted structured plan is then discarded and restricted replanning is triggered. A new candidate plan is generated and the plan's legality is checked again. If a legal plan cannot be obtained within the preset number of replanning attempts, the execution of the current sub-target task is abandoned and reported to the upper layer for task decomposition. The legality of state transitions is represented as: in, and These represent the current security state and the next security state, respectively. Indicates the current event; Represents a predefined set of legal and safe migrations; State bus consistency include: in, The result of the state consistency determination at time t is represented; M represents the set of modules participating in state synchronization. This indicates the current version number of module m; This indicates the reference version used in this round of planning; Indicates the maximum allowed version deviation; if If the consistency requirement is not met, new SkillCalls will be prohibited from being issued.

5. The closed-loop skill scheduling method for a robot agent according to claim 1, characterized in that, Step S6 includes: Instantiate the validated restricted structured plan into a BT, FSM, or SkillRunner execution graph; among them, FSM is suitable for formal conditional branching and state transitions; BT is suitable for concurrent monitoring, reactive scheduling, and recovery subtrees; SkillRunner is suitable for sequential long-chain tasks, row-level retries, and block-level rollbacks; When the execution graph is BT, construct nodes including sequence nodes, selection nodes, retry nodes, timeout nodes, condition nodes, action nodes, parallel monitoring nodes, and recovery nodes / abnormal recovery nodes; When the execution graph is SkillRunner, the execution state completes state_completion row expansion, row-level retries, and BlockTrie block-level rollbacks. When the execution graph is an FSM, legal transitions are performed according to the set of states and the set of events.

6. The closed-loop skill scheduling method for a robot agent according to claim 1, characterized in that, Step S7 includes: SkillFactory / SkillExecutor, acting as a unified skill output, converts SkillCalls generated by FSM, BT, or SkillRunner into specific robot skill calls; wherein, SkillFactory is responsible for determining the skill number... The specific skill instance is constructed based on the skill call parameters bound to the query results of the scene graph; the SkillExecutor is responsible for scheduling the execution of the current skill instance and collecting the execution results.

7. The closed-loop skill scheduling method for a robot intelligent agent according to claim 1, characterized in that, Step S8 further includes: Receive execution feedback from the controller and sensors, including trajectory error, collision risk, gripper current, six-dimensional force, contact normal, object state change, visual relocation result, and controller alarm; convert the feedback into skill execution result SkillResult and structured error code, and write them to the Blackboard; The SkillResult includes: a success flag, error code, telemetry data, object state increment, and recovery strategy suggestion. The Blackboard is used to record skill execution status, error codes, scene changes, node status, and execution progress, and is used to update context information; The behavior tree node status score is calculated based on controller feedback and sensor feedback. This score is used to quantify the execution health of each behavior tree node and serves as the basis for the behavior tree to return a success, failure, or running status, as well as to trigger node retry, subtree recovery, or safety interruption. The behavior tree node state score is represented as follows: in, This represents the execution score of behavior tree node n at time t; Indicates controller feedback; Indicates sensory feedback; Indicates the degree of satisfaction of the postcondition; This indicates a penalty for exceeding the time limit; Indicates a penalty for a dangerous situation; to These are the weighting coefficients; The blackboard summary is represented as follows: in, This represents the compressed summary of the blackboard within the time window. For compression operation, Assign skill numbers, This serves as a marker for successful skill execution. For structured error codes, For incremental changes in the scene, The time sequence of skill execution within the time window [tT,t].

8. The closed-loop skill scheduling method for a robot agent according to claim 1, characterized in that, Step S9 includes: The priority of recovery actions is represented as follows: in, The priority of the recovery action r is indicated; Recoverable(err) indicates the degree to which the current error is recoverable; SuccessHist(r) indicates the historical success rate of the recovery action; Delay(r) indicates the recovery time; Risk(r) indicates the additional risk introduced by the recovery action. to These are the weighting coefficients; When partial recovery is insufficient to resolve the issue, the decision to trigger restricted replanning is based on the cost of replanning. in, This indicates the cost score for triggering a replanning process; Indicates the depth in the task tree where the failure occurred; This indicates uncertainty in the state; Indicates the magnitude of environmental change; This indicates the feasibility of partial restoration; to These are the weighting coefficients.

9. The closed-loop skill scheduling method for a robot intelligent agent according to claim 1, characterized in that, The method further includes: The deployment trajectory quality score is expressed as follows: Where Q(τ) represents the deployment trajectory quality score; SuccessRate(τ) represents the success rate of subtasks in the trajectory; Interrupt(τ) represents the number of multimodal interruptions; and LatencyGap(τ) represents the deviation of the actual time consumption from the end-to-end SLA budget. When deploying trajectory quality scoring When a preset threshold is met, a structured summary of the current trajectory is written into the historical execution experience. Trajectories that do not meet the threshold are not written to ensure the quality of the historical prior.

10. A closed-loop skill scheduling system for a robot intelligent agent, characterized in that, include: Module M1: Receives the user's target task request and simultaneously obtains the robot's own status, environmental status, and task status; The robot's own state, environmental state, task state, historical execution experience, skill registry, and safety rules are all integrated into a unified intelligent agent context; Module M2: Based on the VLM meta controller, the target task is decomposed into multiple sub-target tasks, and the constraints, success criteria and corresponding risk points of each sub-target task are identified. For each sub-target task, a set of candidate skills is initially obtained by screening from the skill registry. Module M3: Based on the robot's own state and the environment state, perform basic qualification verification on each skill in the candidate skill set to meet the preset requirements; only perform feasibility scoring and ranking on the candidate skills whose qualifications meet the preset requirements, and select the one with the best score as the target skill for the sub-target task; If the qualified candidate skill set for a certain sub-target task is empty, then the current execution of the current sub-target task is abandoned and a restricted replanning is triggered. Module M4: Based on the sub-target task and its corresponding target skill, scene graph query results, state summary, and preset template constraints, a restricted structured plan is generated through the VLM meta controller; wherein, the state summary includes: robot's own state, environment state, and task state summary; the scene graph query results are the task object-task object attribute-task object spatial relationships obtained by performing a structured query on the current scene graph for the sub-target task; Module M5: Performs a validity check on the generated restricted structured plan to ensure it meets preset requirements; Module M6: Selects the corresponding structured execution backend based on the characteristics of the sub-target task, and instantiates the validated restricted structured plan into the corresponding execution graph; Module M7: Converts the execution diagram into standardized skill call instructions through a unified skill output, and drives the robot to execute the corresponding skill after being converted by the underlying control. Module M8: Real-time acquisition of execution feedback from controllers and sensors, conversion into structured execution results and error codes, and writing to the shared blackboard, synchronously updating the unified agent context; Module M9: Determines the task execution status based on execution feedback. If execution is normal, the task is advanced until completion. If an execution exception or security event occurs, the local recovery strategy is executed first. If local recovery is ineffective, the VLM restricted replanning is triggered and the execution graph is updated.

Images