Semantic execution of motor tasks, apparatus and storage medium

Abstract

This application relates to the field of robot motion control, and provides a semantic execution method, apparatus, and storage medium for motion tasks. The method includes: acquiring a task structure, wherein the task structure includes semantic representation units (SMUs) and semantic relations (SRs); receiving planning intent information, and mapping the planning intent information to new SMUs and new SRs conforming to the task structure format through an AI planning intent mapping (AIM) interface to update the task structure; driving scheduling based on semantic satisfaction conditions (SSCs), and cyclically executing steps S31 and S32 until all SMUs are executed or the task is terminated; executing the motion operations defined by the SMUs distributed by the SCDE, and updating the state of the SMUs after execution to trigger recalculation and state propagation of the semantic satisfaction conditions in the task structure.

Description

Technical Field

[0001] This application relates to the field of robot motion control, and in particular to a semantic execution method, apparatus and storage medium for motion tasks. Background Technology

[0002] With the widespread application of embodied intelligent systems such as robots, autonomous vehicles, and intelligent agents in complex scenarios such as manufacturing, service, and healthcare, the tasks they need to perform (e.g., grasping, assembly, navigation, human-machine collaboration) are becoming increasingly complex and dynamic. These tasks typically consist of a series of motion operations with specific temporal sequences, conditions, or resource constraints, and require online responses to new instructions or abnormal situations in dynamically changing environments. How to efficiently, flexibly, and robustly model, plan, and schedule these complex motion tasks in real time has become crucial for improving the autonomy and adaptability of embodied intelligent systems.

[0003] In existing technologies, the control of robot motion tasks often employs methods based on pre-programmed sequences, finite state machines, or behavior trees. For example, tasks with fixed processes are executed by pre-programming precise sequences of joint or end-effector trajectories; or task logic is managed by defining states and state transition conditions. These methods can achieve deterministic task execution to a certain extent. However, as task complexity increases, these methods often face the following significant limitations: 1) Their task representation (e.g., fixed code, state nodes) is usually tightly coupled with specific control logic, lacking a unified, parsable semantic description; 2) Existing methods are insufficiently adaptable to dynamic changes during runtime (e.g., adding subtasks, sudden exceptions, changes in conditions); 3) Complex constraints between different motion operations (e.g., timing dependencies, resource mutual exclusion, condition triggering) are usually hard-coded in the control logic, lacking explicit, dynamically evaluable representations, making the task scheduling process rigid and unable to make flexible, condition-driven decisions based on real-time system states. Summary of the Invention

[0004] This application provides a semantic execution method, apparatus, and storage medium for motion tasks. Through semantic modeling and AI mapping, dynamic and condition-driven task scheduling is achieved, which improves the system's execution efficiency and adaptability in complex dynamic scenarios.

[0005] On the one hand, this application provides a semantic execution method for motion tasks, the method comprising: Step S1: Obtain the task structure, which includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU represents a minimum schedulable execution semantic unit of motion operations, and the SR defines the semantic association between at least two SMUs in terms of execution timing or constraints. Step S2: Receive planning intent information, and map the planning intent information into new SMU and new SR that conform to the task structure format through the AI ​​planning intent mapping AIM interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences. Step S3: Based on the semantic satisfaction condition, SSC-driven scheduling is executed repeatedly in steps S31 and S32 until all SMUs have been executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the state of pending execution. For the current SMU, check whether all its incoming SRs meet the triggering conditions. If they all meet the conditions, determine that the semantics of the SMU meet the SSC condition and update its state to ready. Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling and execution engine SCDE for scheduling and distribution; Step S4: Execute the motion operation defined by the SMU distributed by the SCDE, and update the state of the SMU after execution to trigger the recalculation and state propagation of the semantic satisfaction conditions in the task structure.

[0006] On the other hand, this application provides a semantic execution device for motion tasks, the device comprising: The acquisition module is used to acquire the task structure, which includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU is used to represent a minimum schedulable execution semantic unit of motion operation, and the SR is used to define the semantic association between at least two SMUs in terms of execution timing or constraints. The mapping module is used to receive planning intent information and map the planning intent information into new SMU and new SR that conform to the task structure format through the AI ​​Planning Intent Mapping (AIM) interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences. The execution module is used to drive scheduling based on semantic satisfaction condition SSC, and repeatedly execute the following steps S31 and S32 until all SMUs are executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the pending state. For the current SMU, check whether all its incoming SRs meet the triggering condition. If they all meet the condition, it is determined that the semantic satisfaction condition SSC of the SMU is met, and its state is updated to ready. Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling execution engine SCDE for scheduling and distribution. The update module is used to execute the motion operation defined by the SMU distributed by the SCDE, and update the state of the SMU after execution to trigger the recalculation and state propagation of the semantic satisfaction conditions in the task structure.

[0007] Thirdly, this application provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the technical solution of the semantic execution method for motion tasks as described above.

[0008] Fourthly, this application provides a storage medium storing a computer program, which, when executed by a processor, implements the steps of the semantic execution method for the motion task described above.

[0009] As can be seen from the technical solution provided in this application, on the one hand, by introducing Semantic Representation Units (SMUs) and Semantic Relations (SRs) to construct the task structure, a unified and structured description of any complex motion task at the semantic level is achieved. Each SMU encapsulates a minimum schedulable motion operation semantic unit, while the SR explicitly defines the temporal, conditional, and other semantic relationships between SMUs. This allows complex task logic to be decoupled from specific control code, becoming a data structure that can be parsed, modified, and reasoned. Based on this task structure, through the AI ​​Planning Intent Mapping (AIM) interface, diverse planning intent information (e.g., natural language instructions) can be received and automatically mapped and converted into standardized SMUs and SRs, thereby seamlessly integrating the planning capabilities of high-level AI. This enables the system to directly understand and execute high-level, unstructured task instructions, greatly enhancing the system's intelligence and adaptability. On the other hand, through a mechanism based on Semantic Satisfaction Conditions (SSC) driven scheduling, dynamic and intelligent management of task execution is achieved. During execution, the system continuously traverses the task structure and checks each... Whether a task can be scheduled is determined by whether all incoming SRs of each SMU to be executed meet the triggering conditions. This mechanism makes the execution flow of tasks no longer dependent on a preset fixed order, but dynamically determined by the semantic relationships between SMUs and their current satisfaction status. As a result, the system can flexibly handle tasks with complex dependencies and conditional branches, and can automatically and efficiently re-evaluate the executable status of the entire task flow when some task modules fail to execute, external conditions change, or the task structure is updated online, realizing true condition-driven and event-driven dynamic scheduling. Thirdly, the Semantic-Driven Scheduling Execution Engine (SCDE) uniformly schedules and distributes SMUs that meet the conditions, and triggers the recalculation and state propagation of the semantic satisfaction conditions in the task structure after each SMU is executed. This forms a closed-loop execution-feedback-rescheduling control loop. This loop ensures that changes in the task execution status (e.g., completion, failure) can affect the executability evaluation of other related SMUs in real time and automatically according to the definition of SR, thereby making the task execution process of the entire system highly responsive and coherent. In summary, the technical solution of this application can effectively solve the problems of rigid task representation, difficulty in integration with AI systems, and insufficient dynamic adaptability in the prior art, and significantly improve the intelligence level and robustness of embodied intelligent systems in performing complex and dynamic motion tasks. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of the semantic execution method for motion tasks provided in the embodiments of this application; Figure 2 This is a schematic diagram of the structure of the semantic execution device for motion tasks provided in the embodiments of this application; Figure 3 This is a schematic diagram of the device provided in the embodiments of this application. Detailed Implementation

[0012] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0013] In this specification, adjectives such as "first" and "second" are used only to distinguish one element or action from another, without necessarily requiring or implying any actual such relationship or order. Where circumstances permit, reference to an element or component or step (etc.) should not be construed as being limited to only one of the elements, components, or steps, but may be one or more of the elements, components, or steps, etc.

[0014] For ease of description, the dimensions of the various parts shown in the accompanying drawings are not drawn to actual scale.

[0015] In existing technologies, the control of robot motion tasks often employs methods based on pre-programmed sequences, finite state machines, or behavior trees. For example, tasks with fixed processes are executed by pre-programming precise sequences of joint or end-effector trajectories; or task logic is managed by defining states and state transition conditions. These methods can achieve deterministic task execution to a certain extent. However, as task complexity increases, these methods often face the following significant limitations: 1) Its task representation (e.g., fixed code, state nodes) is usually tightly coupled with specific control logic and lacks a unified, parsable semantic description. This makes it difficult for the task model to be directly understood, generated or modified by advanced AI planning systems (e.g., large language models, task planners), forming a "semantic gap". 2) Existing methods are not adaptable enough to deal with dynamic changes during runtime (e.g., adding subtasks, sudden exceptions, changes in conditions). Modifying the task flow often requires reprogramming or adjusting the state machine structure, making it difficult to achieve semantic-based, fine-grained online replanning and real-time adjustment. 3) Complex constraints between different motion operations (e.g., timing dependencies, resource mutual exclusion, condition triggering) are usually hard-coded in the control logic, lacking explicit and dynamically evaluable representations, which makes the task scheduling process rigid and unable to make flexible, condition-driven decisions based on the real-time system state.

[0016] Therefore, there is an urgent need in this field for a unified semantic modeling of complex motion tasks, and a semantically driven, dynamically schedulable task execution scheme, in order to bridge the gap between high-level AI planning and low-level motion control, and improve the system's adaptability and robustness in dynamic environments.

[0017] To address the aforementioned problems in the prior art, this application proposes a semantic execution method for motion tasks, which can be applied to complex task scenarios performed by embodied intelligent systems (e.g., embodied intelligent robots). Its flowchart is attached. Figure 1 As shown, it mainly includes steps S1 to S4, which are detailed below: Step S1: Obtain the task structure, which includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU represents a minimal schedulable motion operation semantic unit, and the SR defines the semantic association between at least two SMUs in terms of execution timing or constraints.

[0018] This step is the starting point and foundation for task execution. The task structure is a formal description of a complete motion task. It does not depend on any specific programming language or controller instructions, but consists of two basic elements: Semantic Modeling Unit (SMU) and Semantic Relation (SR), which will be explained one by one below.

[0019] The Semantic Unit (SMU) is the atomic operation unit of the entire framework. Each SMU encapsulates a motion operation with a clear and complete semantics. For example, it could be "the robotic arm joint J1 moves to position A," "the gripper closes," "wait for 1 second," or "a red object is detected." The key to the concept of "motion operation semantic unit" lies in the word "semantic": each SMU not only defines "what to do" (action) but also defines "with what semantics" through its type (unit_type) and parameter specification (action_spec). This is fundamentally different from a simple function call in traditional code because the semantics of an SMU are self-describing and can be parsed by the scheduling engine. An SMU typically contains fields such as a unique identifier (unit_id), type (unit_type, such as MOVE, WAIT, SYNC, etc.), action specification (action_spec), execution state (exec_state), and constraints. Preferably, a complete SMU data structure may further include: a `resource_requirement` field specifying the hardware or logical resources required for execution, an `executor_binding` field binding to a specific physical or simulated executor, and a `recovery_path` field defining the post-execution failure handling strategy. The specific content of the `action_spec` field is entirely determined by the `unit_type`, forming an extensible semantic system. For example, for an SMU with `unit_type` of `MOVE`, its `action_spec` might contain parameters such as target pose (a set of coordinate values), percentage of motion velocity, acceleration, and trajectory interpolation method (e.g., straight line, arc); for an SMU with `unit_type` of `WAIT`, its `action_spec` might contain a time value or a conditional description of waiting for an external event (e.g., sensor signal). Through this design, motion operations of any granularity can be described in a standardized manner.

[0020] Semantic Relationships (SRs) act as the "glue" connecting SMUs and constructing complex task logic. They explicitly define the semantic relationships between SMUs in terms of execution timing or constraints, including dependencies, triggers, synchronization, or constraint relationships, etc. Here, "semantic relationships in terms of execution timing or constraints" means that the SR describes the rules of the task flow, rather than the specific execution order. Preferably, an SR data structure can contain: a unique relation identifier (relation_id), a relation type (relation_type), a set of source units (source_units), and a set of target units (target_units). For types such as COND (Conditional Trigger), the SR also needs to contain specific trigger rules (trigger_rule) or condition expressions (condition_expr); for RES_MUTEX (Resource Mutual Exclusion) types, it contains the identifier of the contested resource (resource_id). Furthermore, the SR can define control parameters such as priority and timeout, as well as conditional aggregation operators (e.g., "AND" and "OR") used when handling complex logic (such as synchronization) of multiple source units. For example, an SR can declare that "SMU-A can only start after SMU-B is completed" (sequential dependency, SEQ); or "SMU-C and SMU-D must start executing simultaneously" (parallel startup, PAR); or "SMU-E, SMU-F, and SMU-G must all be completed before SMU-H can start" (synchronization barrier, SYNC). Furthermore, SRs can express more complex logic, such as conditional triggering (COND, triggered when a certain condition is met) and resource mutual exclusion (RES_MUTEX, two SMUs cannot occupy the same resource simultaneously). The existence of SRs transforms task logic from an implicit, hard-coded sequence in the program flow to an explicit, dynamically evaluable data relationship. The task structure can be represented in memory as a directed graph, where nodes are SMUs and edges are SRs. However, this application is not limited to graph structures; any data organization form that can effectively store and query SMUs and SRs is acceptable.

[0021] Step S2: Receive planning intent information, and map the planning intent information into new SMU and new SR that conform to the task structure format through the AI ​​planning intent mapping AIM interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences.

[0022] In traditional systems, a significant gap exists between the output intent of an AI planner and the executable control instructions, often requiring tedious manual translation and programming. For example, engineers manually interpret the AI's planning intent and then manually write the corresponding robot control program (e.g., scripts, state machines). This approach is extremely inefficient, cannot achieve automated closed-loop processing, heavily relies on human experts, and is slow to respond, making it unsuitable for dynamic scenarios requiring online replanning. Even with predefined, limited task templates, where the AI ​​output must strictly match template keywords to generate coarse instructions, it still suffers from extremely poor flexibility, can only handle a few preset task patterns, cannot express complex and novel task logic, and has weak generalization ability. Therefore, to address the "semantic gap" and "format mismatch" problems between the output intent of high-level AI planning systems (e.g., large language models) and the underlying executable motion control instructions, this paper proposes a solution. The technical solution adopted in this application is to receive planning intent information and map it into new SMUs and new SRs conforming to the task structure format through the AI ​​Planning Intent Mapping (AIM) interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences. These planning intent information sources include AI planning systems, human-computer interaction interfaces, task scripts, trajectory planners, and / or external control systems, etc. This solution transforms fuzzy, unstructured intents into data objects with clear semantics and structure that can be directly processed by the scheduling engine (SCDE), achieving parsing and executability of task intents. Simultaneously, it enables various AI planners to integrate with the control system in a "plug-and-play" manner, without needing to understand the underlying control details, significantly reducing the engineering threshold for combining AI with embodied intelligence systems, thereby improving the system's intelligent access capabilities and adaptability. Because the mapping is automated, the system can respond in real time to new task instructions generated online by AI, providing a core technical path for achieving true online task planning and adjustment.

[0023] It should be noted that in the above embodiments, the source of the planning intent information includes an AI planning system, indicating that this application is aimed at upper-level intelligent systems, such as embodied intelligent robots. The form of the planning intent information is diverse to adapt to the output habits of different AI systems, and it includes any one or more of natural language instructions, structured data, and motion trajectory sequences, as detailed below: 1. Natural Language Instructions: For example, a user says or types, "Please use the robotic arm to pick up the cup on the table and put it in the cabinet." This is a highly abstract, unstructured expression of intent.

[0024] 2. Structured data: For example, a JSON object describing {"action":"pick_and_place","object":"cup","source":"table","destination":"cabinet"}. This already contains a somewhat parsed intent.

[0025] 3. Motion trajectory sequence: For example, a series of discrete pose points generated by a path planning algorithm. This is closer to the execution layer, but still lacks task-level logical semantics, such as conditions, parallelism, etc.

[0026] The core task of the AIM interface is to uniformly "compile" these heterogeneous inputs into the standard task structure (SMU+SR) defined in step S1. This process is not a simple string matching, but a semantic mapping process that includes understanding, decomposition, and reconstruction.

[0027] As an embodiment of this application, mapping planning intent information into new SMUs and new SRs conforming to the task structure format through the AI ​​planning intent mapping AIM interface can be achieved through the following steps S21 to S23: Step S21: Analyze the planning intent information and identify the motion elements, logical relationships between actions, and constraints.

[0028] This is the preprocessing stage of the mapping. For natural language, Natural Language Processing (NLP) techniques might be used for entity recognition, relation extraction, and intent classification to identify core verbs (e.g., "pick up," "put down"), the object of the operation ("cup"), the location ("on the table," "in the cabinet"), and implicit relations ("then" indicates sequence). For structured data, its fields are directly parsed. For trajectory sequences, it might be necessary to identify the basic action units represented by the trajectories (e.g., "move to the grab point," "perform the grab," "move to the placement point"). Simultaneously, any explicit or implicit constraints (e.g., "gently" might map to a velocity constraint) will also be identified.

[0029] Step S22: Based on the preset mapping rule library, instantiate the identified motion action elements into SMUs with specific unit_type and action_spec.

[0030] The mapping rule base is the knowledge base of the AIM interface, defining how to transform identified action elements into specific SMUs. For example, the rule base might contain: the verb "pick up" -> create an SMU with unit_type MOVE (moves to above the object), followed by another SMU with unit_type MOVE (controls gripper closure); the noun "cup on the table" -> transform the "on the table" and "cup" information into target pose parameters in the action_spec of the first MOVE type SMU (may need to be combined with a scene database). "SMUs with specific unit_type and action_spec" means that each generated SMU is a complete entity populated with specific parameters and can be directly deployed for scheduling.

[0031] Step S23: Based on the identified logical relationships and constraints, generate SRs that connect to the corresponding SMUs. The relation_type of the SR includes at least one of sequential dependency (SEQ), parallel start (PAR), synchronization barrier (SYNC), and condition triggering (COND).

[0032] After generating the SMUs, they need to be organized using SRs according to the logical relationships identified in step S21. The identified "then" relationship will generate a SEQ type SR, connecting the two preceding and following action SMUs. If the intention is "to move the left and right arms simultaneously," a PAR type SR will be generated, where source_units may be a virtual initiation node, and target_units are the MOVE action SMUs of the left and right arms. If there is a condition such as "grasping only when the sensor detects an object," a COND type SR will be generated. In the embodiments of this application, "SR connecting the corresponding SMUs" precisely describes the role of the SR: it establishes directed links between SMUs, thereby weaving independent action units into a task graph with a complete workflow.

[0033] Through steps S21 to S23, a high-level, potentially ambiguous, AI planning intent is automatically and reliably transformed into a specific task structure composed of standardized SMUs and SRs, which can be precisely scheduled and executed by the subsequent engine, thereby updating the system's list of tasks to be executed. This process solves the problem of difficult AI instruction implementation and achieves end-to-end automation from intelligent intent to physical execution.

[0034] After successfully mapping the AI ​​planning intent to a standard task structure (consisting of SMU and SR) in step S2, the system enters the core execution scheduling phase. Traditional task execution models, such as linear scripts or finite state machines, have their execution paths statically predetermined. This rigid scheduling mode is inadequate when facing tasks requiring conditional judgments, parallel collaboration, or runtime exceptions; any deviation from the preset path can lead to process freezes or the need for complex exception handling code. Therefore, this application introduces a novel paradigm of semantic-driven scheduling.

[0035] Step S3: Based on the semantic satisfaction condition, drive the SSC to schedule and repeatedly execute the following steps S31 and S32 until all SMUs have been executed or the task is terminated.

[0036] This step is the central nervous system of the entire method. The core idea of ​​this application is SSC-driven scheduling based on semantic satisfaction conditions. This means that task progression is no longer controlled by fixed, hard-coded sequential code, but by a component called the Semantic Condition Driven Engine (SCDE). This engine dynamically and continuously evaluates the start-up permission conditions (SSCs) of each SMU in the entire task structure graph and schedules only those SMUs whose conditions are met. This forms an active "evaluation-execution-re-evaluation" loop. In a more efficient preferred implementation, SCDE employs an event-driven and local recalculation mechanism. The system establishes and maintains an event queue, an inbound edge index (recording all SRs with each SMU as target_units) and an outbound edge index (recording all SRs with each SMU as source_units) for quickly querying SMU dependencies, and a ready SMU queue. SCDE runs on an event listening loop. When events such as SMU state changes, sensor events, resource releases, security events, or task structure updates occur, the engine does not traverse the entire graph. Instead, it uses the aforementioned inbound and outbound edge indices to precisely locate the set of SMUs directly affected by the event and only recalculates the semantic satisfaction condition (SSC) for the SMUs within that set.

[0037] Step S31: Traverse the SMUs in the task structure that are in the pending state. For the current SMU, check whether all its incoming SRs meet the triggering conditions. If they all meet the conditions, determine that the semantics of the SMU meet the SSC condition and update its state to ready.

[0038] In event-driven mode, step S31 is triggered by an event and executed on the affected set of SMUs. The SCDE engine periodically or under the trigger of a specific event (e.g., a change in the state of an SMU) scans all SMUs in the PENDING state. For each SMU to be executed (denoted as u), SCDE needs to decide: Can u start execution now? The sole basis for this decision is to check whether all its incoming SRs meet the triggering conditions.

[0039] In the above embodiments, the incoming SRs refer to all semantic relationships in the task structure graph with u as the target_unit. These SRs define all the preconditions that u must satisfy before execution can begin. Triggering conditions refer to the specific rules required for each incoming SR to be "activated" or "satisfied." These rules are dynamically determined based on the SR's relation_type. For example: For a SEQ (Sequential Dependency) type SR, the trigger condition is that the execution status of all SMUs in the source_units of the SR is completed. This means that the "sequential dependency" constraint is only released when all preceding SMUs have been successfully executed.

[0040] For a SYNC (synchronization barrier) type SR, the trigger condition determination rule is similar to that of SEQ, which is that the execution status of all SMUs in the source_units of the SR is completed. However, this is usually used to coordinate multiple parallel branches to ensure that they all reach a convergence point.

[0041] For a Resource Exclusion (RES_MUTEX) type SR, the trigger condition is that all System Units (SMUs) in the source_units of the SR have released their holdings of the specified resource. For example, if SMU-A in source_units holds the robotic arm, then SMU-B (target_units) must wait for SMU-A to release the robotic arm before its corresponding RES_MUTEX relationship is satisfied.

[0042] The SCDE will calculate whether each incoming SR of u is satisfied according to the above rules. The statement "If all are satisfied, then the semantics of the SMU satisfy the SSC condition" is an AND operation. Only when all incoming SRs of u (representing all pre-constraints) report "satisfied" is u's global start permission (SSC) considered valid. At this point, the SCDE updates u's exec_state from PENDING to READY, indicating that it has gained execution eligibility and enters the ready queue to await dispatch. This process is essentially a declarative, data dependency-based scheduling decision. It makes the scheduling engine's decision-making process clear and deterministic, enabling precise handling of complex scenarios such as parallelism, synchronization, and resource contention, ensuring system security and orderly resource coordination during concurrent execution of multiple tasks.

[0043] It should be noted that for a COND (Conditionally Triggered) type SR, its triggering condition requires not only the completion of source_units, but also the fulfillment of a specific logical expression or event configured in its condition field (e.g., sensor values ​​reaching a threshold). In other words, to ensure that condition-based task triggering does not rely on a one-time initial judgment but can continuously respond to dynamically changing system states or environmental signals, when the generated SR's relation_type is COND, Figure 1 The example method may also include: configuring trigger conditions based on sensor signals, system state variables, or logical expressions in the condition field of the SR; continuously monitoring the trigger conditions during scheduling, and determining that the COND type SR meets the trigger conditions only when the trigger conditions are met. This scheme achieves true "event-driven" scheduling, enabling the task flow to respond in real time to changes in sensor data or system events, greatly enhancing the system's adaptability and responsiveness in uncertain environments.

[0044] Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling execution engine SCDE for scheduling and distribution.

[0045] Once the SMU enters the READY state, the SCDE will remove it from the ready queue and schedule and distribute it. "Submitting to the semantically driven scheduling and execution engine SCDE for scheduling and distribution" means that the SCDE, as a unified coordinator, is responsible for allocating executable semantic units to appropriate execution backends. This may involve: placing the SMU into an internal execution queue; invoking the corresponding executor adaptation module (see step S4); and updating the SMU's state from READY to RUNNING. The SCDE's scheduling strategy can be a simple FIFO (First-In-First-Out) or a more complex algorithm based on priority, resource optimization, etc., but all of these are within the scope of this application's protection because the core scheduling basis (SSC) has already been determined in step S31.

[0046] As can be seen from the cyclic execution of steps S31 and S32 in the above embodiment, it perfectly solves the scheduling rigidity problem caused by the solidified execution logic of traditional task scheduling methods (e.g., sequential queues, finite state machines). On the one hand, it realizes true condition-driven and event-driven scheduling, that is, the progress of tasks no longer depends on a preset fixed path, but is dynamically determined by data flow and event flow (manifested as the satisfaction of SR conditions), which makes handling conditional branches, abnormal waiting, resource contention and other scenarios natural and direct. On the other hand, it endows the system with inherent concurrent coordination capabilities. This is because as long as the incoming SR conditions of multiple SMUs are satisfied simultaneously, they can be scheduled in parallel, and complex synchronization requirements (e.g., synchronization after multi-axis motion) can be elegantly expressed by a SYNC type SR and automatically decided by this mechanism without the need for complex intermediate control logic. Thirdly, it enables runtime dynamic update operations. The scheduling engine provides a foundation for Operation (DUO) and robustness because scheduling decisions are calculated in real time based on the current task structure and SMU state. Therefore, when a task structure is inserted, deleted, or modified during runtime, the scheduling engine only needs to recalculate the SSC once to automatically adapt to the new task graph. This natively supports online replanning. Similarly, when an SMU fails to execute, the SSC of its subsequent SMUs will not be satisfied, thus automatically blocking and providing a safe context for executing error recovery strategies.

[0047] Considering that in a dynamic environment, the execution of an SMU is not an isolated event, and changes in its state, resource consumption, or time may affect other SMUs, this application introduces a constraint propagation mechanism as an important component of SSC-driven scheduling to efficiently and accurately assess the impact of changes in the execution parameters or state of a particular SMU on the feasibility of the global task, and to avoid global deadlock or safety violations caused by local changes. This mechanism can be implemented through steps S33 and S34, detailed below: Step S33: When a change in the execution time, resource occupancy status, or security status of any SMU is detected, according to the definition of SR, the constraint update event caused by the change is propagated to other SMUs that have a direct or indirect semantic relationship with it.

[0048] For example, if an SMU times out, this "time constraint violation" event will propagate through a TIMEOUT type SR, triggering its target_units (which may be an exception handling SMU). Propagating the constraint update event caused by the change to other SMUs with direct or indirect semantic association includes the security constraint propagation example in steps S331 to S333 as follows: Step S331: When the system's global security state is triggered (e.g., the emergency stop button is pressed, or the system enters a protected area), a security constraint update event is generated.

[0049] In the above embodiments, the system global safety state refers to the safety status affecting the entire system or its critical components, typically triggered by hardware safety loops, safety monitoring software, or operators. Triggering the system global safety state means the system has transitioned from a safe state to a non-safe or warning state, such as when an emergency stop signal (E-Stop) is active, a safety light curtain is obstructed, joint torque exceeds limits, or the system enters a preset prohibited area. When such events occur, a dedicated safety monitoring module generates a safety constraint update event. This event is a structured message that includes at least the event type (e.g., EMERGENCY_STOP), the triggering source, and a timestamp indicating when the event took effect. This event is broadcast to the scheduling execution engine SCDE.

[0050] Step S332: Iterate through all SMUs in the ready or running state and check whether their action_spec or constraints fields contain constraints that conflict with the current security state.

[0051] Upon receiving a safety constraint update event, SCDE immediately initiates a global scan. "Traversing all SMUs in the ready or executing state" is crucial because these SMUs are units that are about to or are currently affecting the physical world and must be prioritized. "Checking their action_spec or constraints fields" is key to determining conflicts. Each SMU's constraints field can predefine a series of safety rules, such as "prohibit execution in an emergency stop state" and "maximum speed limit." The action_spec field describes the specific action; for example, the target location of a MOVE action might be within a restricted area in the current safety state. The checking process compares the safety event type with the constraint rules or action parameters defined in the SMU to determine if there are any logical or spatial conflicts.

[0052] Step S333: For SMUs with conflicts, change their state from ready or running to paused or pending execution, and prevent them from being scheduled or continuing execution until the safe state is lifted.

[0053] For any Conflicting Streaming Management Units (SMUs) detected, the system must immediately implement isolation measures. A critical operation for conflicting SMUs is changing their state from Ready or Executing to Paused or Pending: for SMUs in the READY state, their state is rolled back to PENDING; for SMUs in the RUNNING state, a pause command is sent to the executor, and their state is marked as PAUSED. Preventing conflicting SMUs from being scheduled or continuing execution ensures that conflicting actions are immediately frozen, preventing the escalation of security risks. All conflicting SMUs will remain in this blocked state until the safe state is lifted, i.e., a clear safe state recovery event is received. At this point, they may be re-evaluated, and if the constraints no longer conflict, they can be restored to the Ready state.

[0054] Step S34: Based on the constrained update event after propagation, re-evaluate the semantic satisfaction condition SSC and its schedulability of the affected SMU.

[0055] This ensures that any local changes are reflected in the scheduling decisions in a timely and global manner, thus guaranteeing the overall security and consistency of the system.

[0056] The technical solutions exemplified in steps S33 and S34 above achieve global consistency maintenance of system constraints, which can prevent execution failures or security incidents caused by constraint violations, and improve the robustness and security of complex task execution.

[0057] To allow for safe and uninterrupted modification of the task structure during task execution without causing semantic confusion, resource conflicts, or disrupting the results of already executed tasks, Figure 1 The example method may also include performing the following dynamic updates during the execution of step S3, as described in steps S5 to S7: Step S5: Receive dynamic update operation DUO instructions for the runtime task structure. DUO instructions include at least one of inserting, deleting, replacing SMU or SR.

[0058] In the above embodiments, receiving a dynamic update operation DUO instruction for the runtime task structure means that the system exposes a standard interface that allows external entities (e.g., AI planning systems, human-computer interaction interfaces) to send modification commands during task execution. The DUO instruction itself is a structured data object that explicitly specifies the operation type (INSERT, DELETE, REPLACE), the operation object (identifiers and new content of one or more SMUs / SRs), and the target location of the operation (e.g., predecessor and successor relationships). For example, an insert instruction would contain the new SMU / SR to be inserted, as well as the identifiers of the predecessor and successor SMUs that should be connected in the task graph. After receiving, the instruction enters a verification queue. The received dynamic update operation DUO instruction can be generated and sent online by the AI ​​planning system to achieve online replanning or anomaly recovery.

[0059] As a preferred embodiment of transactional processing, the DUO instruction itself can be designed as a structured data object, including but not limited to: a unique identifier for the update operation (update_id), the version number of the task structure on which it is based (base_graph_version), the operation type (operation_type), the target element identifier (target_id), the operation payload (i.e., the SMU / SR data to be inserted / replaced), and optional attachment point information (attach_point).

[0060] Step S6: Before executing the DUO instruction, verify its consistency with the current task structure and execution state to ensure that the execution semantics of the completed SMU are not violated.

[0061] Before applying DUO instructions, rigorous consistency verification should be performed, which is the cornerstone of system robustness. Verification is a collection of checks: First, check the validity of the ID of the operation object to avoid referencing non-existent SMUs. Second, check that the execution semantics of completed SMUs are not violated; this is the most important principle. For example, a new SR cannot be inserted that requires an SMU in a COMPLETED state as its target_units (i.e., requires a completed action to wait for an action that has not yet occurred), as this would create a logical contradiction. Similarly, an SR that is already depended upon by other RUNNING or COMPLETED SMUs cannot be deleted. Finally, it is also necessary to check the validity of the fields of the newly introduced SMU / SR itself, and whether modifications would immediately lead to resource deadlocks or security conflicts. Only after all checks are passed will the instruction be allowed.

[0062] It should be noted that consistency verification is a multifaceted checking process, which may include: checking the validity of target_id (this is to prevent operations on non-existent nodes), checking whether modifications will introduce directed circular dependencies, checking whether newly introduced actions will cause deadlock conflicts with the current resource occupancy state, checking whether the security constraints of new actions conflict with the current system security state, and most importantly—checking whether operations will break the execution semantics of SMUs that have been marked as "completed". For example, a dependency that requires a completed task to wait for a task that has not yet occurred cannot be added.

[0063] Step S7: After passing the consistency verification, apply the DUO instruction to modify the task structure and immediately trigger the recalculation of the semantic satisfaction condition SSC for the affected SMU.

[0064] After successful verification, SCDE applies DUO instructions to modify the task structure in a transactional manner. For insertion, the new SMU / SR is added to the graph data structure in memory; for deletion, the object is removed and its associated links are cleaned up; for replacement, the object is atomically swapped. Immediately after modification, the semantic satisfaction condition SSC for the affected SMUs is recalculated. SCDE locates all directly and indirectly affected SMUs (e.g., all successor SMUs of the newly inserted SMU) based on the region affected by the modification, and then re-executes the SSC calculation process of step S31 for these SMUs. In the transactional implementation, the verification and application of DUO instructions are performed atomically: after successful verification, all modifications are atomically applied and the global task structure version number is incremented; if verification fails or an error occurs during application, the entire update operation is rolled back, and the system state remains unchanged. Since the scheduling core dynamically calculates SSCs, local modifications to the task graph can immediately affect subsequent scheduling through SCDE's retracing, without stopping the entire task, achieving true online replanning.

[0065] The technical solutions in steps S5 to S7 provide the system with the core capability of online replanning, enabling it to cope with unexpected obstacles, task changes, or optimization adjustments without stopping and restarting the entire task, thus significantly improving the system's flexibility and operational efficiency.

[0066] It should be noted that, in order to handle resource contention more precisely and improve system security, this application may also include the following mechanisms: Resource Lease Table: The system maintains a global resource lease table, which records in real time the current occupancy status, occupant (unit_id of the SMU), and lease duration of each physical or logical resource (e.g., robotic arm, end effector, specific work area). When the scheduler calculates the semantic satisfaction condition (SSC) of the SMU, if a Resource Relation (SR) with relation_type of resource mutual exclusion (RES_MUTEX) is involved, this resource lease table will be queried. An SR of type RES_MUTEX is considered to meet the triggering condition only if the resource specified by the SR is in an "idle" state in the lease table, or if its current occupant is the source_unit of that SR and is about to be released. The SMU must register its occupancy in the lease table when it starts execution and register its release after execution.

[0067] Transactional processing of security events: When security events such as emergency stops, collision warnings, or entry into dynamic protection zones occur, the security constraint update events generated by the system can carry higher processing priority. When handling such events, the SCDE can trigger a security transaction: immediately suspend all scheduling decisions under evaluation, prioritize the propagation of security constraints (steps S331 to S333), and put all conflicting SMUs into a suspended or rolled-back state. Within this transaction, the system can also automatically generate and insert a "safety recovery subgraph" (which itself is a DUO instruction) for the affected task branches according to a preset strategy. This subgraph contains the necessary operations to restore the system to a safe state or resume execution from the breakpoint. This mechanism ensures that the security response is not only a passive suspension but also actively guides the system towards a safe state.

[0068] Step S4: Execute the motion operation defined by the SMU distributed by SCDE, and update the state of the SMU after execution to trigger the recalculation of semantic satisfaction conditions in the task structure and state propagation.

[0069] Step S4 is the final semantic-to-physical conversion and execution feedback stage, where the SCDE retrieves an SMU from the ready queue and distributes it. Specifically, the SCDE selects the corresponding actuator adapter based on the information indicated by the SMU's executor_binding field. Executing the motion operation defined by the SMU distributed by the SCDE means that the action described by the SMU is actually completed by a specific physical actuator (e.g., robot joint servo drive, humanoid robot leg motor, CNC machine tool feed axis) or simulator. The execution process requires parsing and adapting the SMU's action_spec. The action_spec field of the SMU contains different parameter sets depending on its unit_type. When unit_type is WAIT, the action_spec field contains waiting conditions, which can be time conditions, sensor signal conditions, or external event conditions. When unit_type is MOVE, the action_spec field contains target pose, motion velocity, and trajectory type parameters. For SMUs with unit_type MOVE, the execution adaptation includes the process illustrated in steps S41 to S43: Step S41: Parse the action_spec field of the MOVE type SMU and extract the target pose and motion velocity parameters.

[0070] Parsing the `action_spec` field of a MOVE-type SMU refers to reading this structured data block, `action_spec`, from the SMU. For the MOVE type, this data block typically contains multiple subfields. The target pose describes the endpoint of the motion; for a robotic arm, this might be the position and orientation in Cartesian space (X, Y, Z, Rx, Ry, Rz); for a mobile robot, it might be the coordinates and orientation on a map; and for a single joint, it might be the angle value. The motion velocity parameter might be a scalar (such as a percentage of maximum velocity) or a complete velocity curve description. The parser accurately extracts these key parameter values ​​from the `action_spec` to prepare for the next step of conversion.

[0071] Step S42: Based on the target actuator type, call the corresponding kinematic solver or control interface to convert the target pose and motion velocity parameters into low-level control instructions that the actuator can recognize.

[0072] The type of target actuator determines which conversion logic is used. For example, for a six-axis collaborative robot, it might be necessary to call an inverse kinematics (IK) solver to convert the target pose into target angle values ​​for the six joints. Simultaneously, the velocity percentage parameter is converted into specific joint angular velocities or torque commands. For a mobile chassis, this might be converted into velocity commands for the left and right wheels. For a simulator, it might be converted into function commands that call its internal API. The low-level control commands that the actuator can recognize are the signals that ultimately drive the hardware or simulator, such as specific CAN messages, EtherCAT commands, or ROS control messages. This step encapsulates all the details related to the specific brand and model. For non-MOVE type SMUs (e.g., GRASP, WAIT), the adapter also parses the parameters in its action_spec (e.g., gripper opening degree, waiting conditions) and converts them into corresponding low-level commands.

[0073] Step S43: Send the underlying control commands to the corresponding physical actuators for execution.

[0074] After the conversion is complete, the adapter layer sends the underlying control commands to the corresponding physical actuators via the appropriate communication bus (e.g., EtherCAT, CAN, ROS Topic, etc.). The actuators receive the commands and begin execution, typically starting a monitoring loop to detect completion, timeouts, or errors.

[0075] After execution (whether successful or not), the executor or adaptation layer reports the result to the SCDE. Upon completion, the SCDE updates the SMU's state from RUNNING to COMPLETED (success) or FAILED (failure). This state change is a critical event, triggering the recalculation and state propagation of semantic satisfaction conditions within the task structure. This means the event immediately "wakes up" the SCDE's scheduling loop (step S3). Because of this SMU state change, some SRs (e.g., SEQ, SYNC) with it as their source_units may directly satisfy the triggering conditions, thus enabling the SSC of these SRs' target_units (subsequent SMUs) to be established, changing their state from PENDING to READY. In this way, the completion signal, like a pebble thrown into water, automatically spreads its ripples (state propagation) through the dependency network built by the SRs, driving the task forward and forming a fully automated, data-driven closed loop.

[0076] If the SMU fails to execute, the following steps S8 and S9 will be triggered as examples of error recovery paths.

[0077] Step S8: When it is detected that the execution status of an SMU has been updated to failure, query the recovery strategy defined in the recovery_path field of that SMU.

[0078] Step S9: Execute the corresponding recovery operation according to the recovery strategy. The recovery operation includes: retrying the current SMU, skipping the current SMU and scheduling its subsequent SMU, reverting to the previous SMU specified in the task structure to start again, or triggering an exception handling subtask process.

[0079] Triggering an exception handling subtask is an advanced recovery method that goes beyond simple retries or skipping. It provides a structured, customizable, and automated advanced recovery approach for complex execution failures. The specific implementation includes the following steps S91 to S93 as an example: Step S91: Load the predefined exception handling subtask template based on the identifier in the recovery_path field.

[0080] When the `recovery_path` field indicates that a subtask flow needs to be triggered, it contains a template identifier (e.g., `Template_Recover_Drop`). Based on the identifier in the `recovery_path` field, the system loads a predefined exception handling subtask template from a pre-stored template library. This template itself is a parameterized, incomplete small task structure containing several SMUs and SRs, but some parameters (e.g., target location, reference object) are marked as variables to be populated.

[0081] Step S92: Based on the context information of the currently failed SMU, parameterize and instantiate the exception handling subtask template to generate a subtask structure containing several SMUs and SRs.

[0082] After loading the template, the system collects the context information of the currently failed SMU, which may include the robot's pose at the time of failure, sensor readings, and failure error codes. This context information is then used to populate the variables in the template. For example, a "fallen object recovery" template might need to use the current position of the robotic arm's end effector to instantiate the search center point for the "scan the ground" action. Once parameterization and instantiation are complete, a subtask structure containing several SMUs and SRs is generated; this is a complete, independently schedulable task segment.

[0083] Step S93: Insert the subtask structure into the current task structure and establish its semantic connection with the original task structure. The subtask structure is then scheduled and executed by SCDE.

[0084] After generating the subtask structure, it needs to be integrated into the main task. Inserting the subtask structure into the current task structure typically means inserting this subtask structure block after the failed SMU and before its original successor SMU. Establishing a semantic connection with the original task structure is crucial, i.e., creating new SRs: connecting from the failed SMU (or a new node representing the failure event) to the entry SMU of the subtask structure (SEQ relationship), and connecting from the exit SMU of the subtask structure to the successor SMU of the original failed SMU (SEQ relationship). In this way, the recovery subtask is seamlessly embedded into the main task flow. Subsequently, the SCDE schedules and executes the subtask structure. Because the SCDE continuously performs SSC calculations, it immediately recognizes that the conditions of the newly inserted subtask entry SMU have been met (because its preceding failure event has occurred), thus automatically starting to schedule and execute the entire recovery process. This is actually an automated, dynamic update (DUO) and replanning for a specific fault type, greatly improving the system's autonomous fault tolerance.

[0085] In practical applications, it is often necessary to save task structures to files, transfer them between different controllers, or seamlessly migrate tasks between simulation environments and real robots (Sim-to-Real). Traditional methods typically save tasks in a scripting language specific to a particular robot brand (e.g., URScript) or in code tightly coupled with control logic, making them unusable across platforms. To address this issue, this application provides a standardized method for task structure persistence and migration, as detailed in steps S10 to S12.

[0086] Step S10: Serialize the complete task structure containing the SMU set and SR set into a platform-independent serialization representation SER.

[0087] In this embodiment, serialization into a platform-independent serialized representation (SER) is the core of step S10. SER is an abbreviation for Serialized Representation, which is a process of encoding a task structure graph (i.e., U (set of SMUs) and R (set of SRs)) in memory into a standardized, linear byte stream or text format. Platform independence is a key attribute of SER, meaning that the SER format does not contain any information related to a specific robot controller, operating system, or programming language runtime. Typically, SER is implemented using a widely supported structured data format, such as JSON, YAML, XML, or Protocol Buffers. The serialization process iterates through all SMUs and SRs, encoding all their core fields (unit_id, unit_type, action_spec, relation_type, source / target_units, etc.) completely into the SER according to predefined format specifications. For example, for a MOVE type SMU, the target pose (a set of floating-point numbers) in its action_spec will be encoded as is, without being converted to a brand-specific controller instruction format. The resulting SER file or data stream becomes a standardized snapshot of the task semantics.

[0088] Step S11: Transmit or store the SER.

[0089] After generating the SER, subsequent operations can be performed flexibly. Transmission refers to sending the SER from one system (such as a cloud planning server or simulation workstation) to another system (such as a real robot controller in a workshop) via communication methods such as networks or buses. Storage involves saving the SER to local hard drives, databases, or cloud storage for task version management, archiving, or offline analysis. Because the SER is platform-independent pure data, the transmission and storage processes do not need to concern themselves with the specific content of the task, greatly simplifying the complexity of inter-system collaboration and data management.

[0090] Step S12: On the target system, the received SER is deserialized to reconstruct a task structure consistent with that of the source system for SCDE scheduling and execution.

[0091] Once the target system (e.g., another robot or simulator) receives the SER, a reverse operation needs to be performed. Deserializing the received SER involves parsing the SER data stream according to the same format specification, restoring it to a set of SMU and SR objects in memory. Reconstructing a task structure consistent with the source system means that the task graph generated in the target system's memory after deserialization is semantically identical to the source system, i.e., the connections and constraints between SMUs are completely consistent. This reconstructed task structure can then be directly submitted to the SCDE engine on the target system for loading and scheduling. Thus, a task achieves transparent cross-platform migration. Tasks verified in simulation can be serialized into SER and then deserialized for execution on a real robot, providing a fundamental guarantee for reliable Sim-to-Real transfer at the task description layer.

[0092] Considering that software systems may iterate and upgrade, the format of SER may also evolve. To ensure that tasks saved in older versions can be correctly read in newer versions of the system, this application introduces a version adaptation mechanism. If the serialized representation SER generated in step S10 contains a version number field, the deserialization of the received SER can be implemented using the schemes exemplified in steps S121 and S122: Step S121: Read the version number field in SER to identify its version.

[0093] In the root node or header of the SER, there is a dedicated version field. When the target system loads the SER file, it first reads the version number field in the SER, for example, "version": "1.2". The parser uses this field to identify its version and determine which version of the serialization specification the SER was generated from.

[0094] Step S122: If the identified version is different from the current version supported by the target system, the corresponding version adapter is called to convert the fields in SER to be compatible with the target system.

[0095] After identifying the version, it is compared with the latest version supported by the target system (e.g., "2.0"). If the identified version differs from the current version supported by the target system, it indicates a potential format incompatibility. In this case, direct parsing is not possible; instead, the corresponding version adapter is invoked. The version adapter is a set of predefined conversion rules or functions specifically designed to map or convert field names, structures, or data types in a specific older version of the SER to the format expected by the new version. For example, if a field in version 1.0 is `max_speed`, and in version 2.0 it is renamed `speed_limit`, the adapter is responsible for performing this renaming operation. After converting the fields in the SER, an equivalent task structure representation conforming to the current system version is obtained, thus ensuring compatibility with the target system. This mechanism guarantees forward compatibility of the task description, ensuring that system upgrades do not cause historical tasks to become invalid.

[0096] As the scheduling hub, the operating mode of the SCDE determines the system's response speed and efficiency. While polling to scan all SMUs is simple, it is inefficient when the task scale is large. This application prefers event-driven as the core operating mode of the SCDE, that is, the semantic-driven scheduling execution engine SCDE runs in an event-driven manner, specifically including the following steps S301 to S303.

[0097] Step S301: SCDE maintains an event listener loop.

[0098] After initialization, SCDE starts an event listening loop. This loop typically consists of an event queue and an event dispatcher. SCDE itself does not actively and frequently traverse the entire task graph; instead, it enters a "standby" state, with its main thread blocked on the event queue, waiting for events to arrive. This design avoids unnecessary CPU idling and uses computing resources at critical moments.

[0099] Step S302: When the execution status of any SMU changes, or a new dynamic update operation DUO is applied, or an external event is triggered, the SCDE receives the corresponding event notification.

[0100] The core of the event loop is receiving corresponding event notifications. The system contains multiple event producers, including the executor adaptation layer, the dynamic update module, and external systems, which are described below: 1. Actuator Adaptor Layer: When an SMU completes or fails to execute, an SMU_STATE_CHANGED event is generated, which includes the SMU's ID and the new state.

[0101] 2. Dynamic Update Module: When the DUO instruction is successfully applied, a TASK_STRUCTURE_MODIFIED event will be generated, along with the affected SMU ID range.

[0102] 3. External systems: such as sensor threshold triggering, changes in safety status, user interrupt commands, etc., will generate corresponding EXTERNAL_EVENT events.

[0103] These events are encapsulated into messages of a uniform format and sent to the SCDE's event queue. The phrase "when the execution state of any SMU changes, or a new dynamic update operation DUO is applied, or an external event is triggered" describes the main sources of events, ensuring that any situation that may affect scheduling decisions is perceived by the SCDE.

[0104] Step S303: In response to the event notification, SCDE locates the affected SMUs and triggers the recalculation process of the semantic satisfaction condition SSC for these SMUs.

[0105] The SCDE event dispatcher retrieves an event from the queue and responds to the event notification. First, it locates the affected SMUs: based on the event type and payload, it quickly determines the set of SMUs that need to be re-evaluated. For example, for the SMU_STATE_CHANGED event, all target_units (i.e., their successor SMUs) pointed to by SRs with that SMU as source_units are affected. Then, it triggers the recalculation process of the semantic satisfaction condition SSC for these SMUs; that is, for this finite set of affected SMUs, it executes the logic of step S31 to check their incoming SR conditions. Because it only needs to compute local rather than global conditions, this event-driven model greatly improves scheduling efficiency, enabling the system to respond to state changes in milliseconds.

[0106] In industrial scenarios, many task processes (e.g., "standard pick and place", "bolt tightening", "visual positioning") are used repeatedly. To avoid repeatedly building the same SMU-SR structure, improve development efficiency, and ensure consistency, this application supports the invocation of reusable subtask templates before step S1, as described in steps S0a to S0d.

[0107] Step S0a: Receive the calling instruction and instantiation parameters for a subtask template.

[0108] Instead of describing the details from scratch, the task builder (which could be an AI system or an engineer) issues a call instruction containing the name of the desired subtask template (e.g., "PickAndPlace") and instantiation parameters. Instantiation parameters are key information used to customize the template; for example, for the "pick and place" template, parameters might include the ID of the target object to be picked up, the coordinates of the placement position, and the gripper speed. This is equivalent to calling a "function" and passing in "actual parameters."

[0109] Step S0b: Based on the calling instruction, obtain the predefined, parameterized subtask template. The subtask template defines the internal SMU set, SR set, and parameter interface.

[0110] The system retrieves a predefined, parameterized subtask template from the template library based on the template name. This template is a saved, partially abstract task structure block. It defines its internal set of SMUs and SRs, describing a complete sub-process (e.g., move upwards -> descend -> close gripper -> ascend -> move to target -> descend -> open gripper). Simultaneously, the template defines a parameter interface, explicitly indicating which SMUs' `action_spec` fields (e.g., target pose) are to be populated. These parameters are typically represented as placeholders (e.g., `{pick_position}`, `{gripper_speed}`).

[0111] Step S0c: Instantiate the subtask template using the instantiation parameters to generate a specific, schedulable subtask structure block.

[0112] Next, the system performs a "compile" operation. Using instantiation parameters, the subtask template is parameterized and instantiated. Specifically, the system iterates through each SMU in the template; if its `action_spec` contains a placeholder with the same name as the parameter interface, it replaces it with the corresponding parameter value provided in the calling instruction. For example, `{pick_position}` is replaced with the specific `[x:0.5,y:0.2,z:0.1]`. After the replacement, the IDs of all SMUs and SRs in the template are reassigned to ensure uniqueness within the current task context. This process ultimately generates a concrete, schedulable subtask structure block, which is an independent, parameterized task subgraph.

[0113] Step S0d: The instantiated subtask structure block is embedded as a whole into the currently constructed task structure, which is presented to the outside as a logical SMU.

[0114] Finally, this instantiated structure block needs to be integrated into a larger main task. There are two integration methods: one is to expand all its internal SMUs and SRs and directly connect them with the main task's SMU / SRs; the other, more efficient method is to embed it as a whole. The essence of this approach is that it appears externally as a logical SMU: the system can create a new SMU of type SUB_TASK (or a "composite SMU"), using its sub_units field to reference the entire instantiated subtask structure block. Then, in the main task graph, only SRs related to this SUB_TASK type SMU (such as sequential dependencies) need to be created, without needing to concern themselves with its complex internal structure. During scheduling, SCDE treats this composite SMU as a black box, activating its internal subtask structure's independent scheduling loop only when its SSC satisfies scheduling requirements. This approach greatly simplifies the construction and management of complex tasks, achieving modularization and hierarchical structure in task description.

[0115] From the above Figure 1As seen in the example of the semantic execution method for motion tasks, on the one hand, by introducing Semantic Representation Units (SMUs) and Semantic Relations (SRs) to construct the task structure, a unified and structured description of any complex motion task at the semantic level is achieved. Each SMU encapsulates a minimum schedulable semantic unit of motion operation, while the SR explicitly defines the temporal, conditional, and other semantic relationships between SMUs. This allows complex task logic to be decoupled from specific control code, becoming a data structure that can be parsed, modified, and reasoned. Based on this task structure, through the AI ​​Planning Intent Mapping (AIM) interface, diverse planning intent information (e.g., natural language instructions) can be received and automatically mapped and converted into standardized SMUs and SRs, thereby seamlessly integrating the planning capabilities of high-level AI. This enables the system to directly understand and execute high-level, unstructured task instructions, greatly enhancing the system's intelligence and adaptability. On the other hand, through a mechanism based on Semantic Satisfaction Conditions (SSC) driven scheduling, dynamic and intelligent management of task execution is achieved. During execution, the system continuously traverses the task structure and checks... Whether each SMU to be executed can be scheduled is determined by whether its incoming SRs meet the triggering conditions. This mechanism makes the execution flow of tasks no longer dependent on a preset fixed order, but dynamically determined by the semantic relationships between SMUs and their current satisfaction status. As a result, the system can flexibly handle tasks with complex dependencies and conditional branches, and can automatically and efficiently re-evaluate the executable status of the entire task flow when some task modules fail to execute, external conditions change, or the task structure is updated online, realizing true condition-driven and event-driven dynamic scheduling. Thirdly, the Semantic-Driven Scheduling Execution Engine (SCDE) uniformly schedules and distributes SMUs that meet the conditions, and triggers the recalculation and state propagation of the semantic satisfaction conditions in the task structure after each SMU is executed. This forms a closed-loop execution-feedback-rescheduling control loop. This loop ensures that changes in the task execution status (e.g., completion, failure) can affect the executability evaluation of other related SMUs in real time and automatically according to the definition of SR, thereby making the task execution process of the entire system highly responsive and coherent. In summary, the technical solution of this application can effectively solve the problems of rigid task representation, difficulty in integration with AI systems, and insufficient dynamic adaptability in the prior art, and significantly improve the intelligence level and robustness of embodied intelligent systems in performing complex and dynamic motion tasks.

[0116] Please see the appendix Figure 2 This application provides a semantic execution device for motion tasks. The device includes an acquisition module 201, a mapping module 202, an execution module 203, and an update module 204, as detailed below: The acquisition module 201 is used to acquire the task structure, wherein the task structure includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU is used to represent a minimum schedulable execution semantic unit of motion operation, and the SR is used to define the semantic association between at least two SMUs in terms of execution timing or constraints. The mapping module 202 is used to receive planning intent information and map the planning intent information into new SMU and new SR that conform to the task structure format through the AI ​​planning intent mapping AIM interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences. The execution module 203 is used to drive scheduling based on semantic satisfaction condition SSC, and repeatedly execute the following steps S31 and S32 until all SMUs are executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the pending state. For the current SMU, check whether all its incoming SRs meet the triggering conditions. If they all meet the conditions, it is determined that the semantic satisfaction condition SSC of the SMU is met, and its state is updated to ready. Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling execution engine SCDE for scheduling and distribution. The update module 204 is used to execute the motion operations defined by the SMU distributed by the SCDE, and update the state of the SMU after execution to trigger the recalculation of semantic satisfaction conditions in the task structure and state propagation.

[0117] From the above Figure 2As seen in the example of the semantic execution device for motion tasks, on the one hand, by introducing Semantic Representation Units (SMUs) and Semantic Relations (SRs) to construct the task structure, a unified and structured description of any complex motion task at the semantic level is achieved. Each SMU encapsulates a minimum schedulable semantic unit for motion operations, while the SR explicitly defines the temporal, conditional, and other semantic relationships between SMUs. This decouples complex task logic from specific control code, making it a data structure that can be parsed, modified, and reasoned about. Based on this task structure, through the AI ​​Planning Intent Mapping (AIM) interface, diverse planning intent information (e.g., natural language instructions) can be received and automatically mapped and converted into standardized SMUs and SRs, thereby seamlessly integrating the planning capabilities of high-level AI. This allows the system to directly understand and execute high-level, unstructured task instructions, greatly enhancing the system's intelligence and adaptability. On the other hand, through a mechanism based on Semantic Satisfaction Conditions (SSCs) driven scheduling, dynamic and intelligent management of task execution is achieved. During execution, the system continuously traverses the task structure and checks... Whether each SMU to be executed can be scheduled is determined by whether its incoming SRs meet the triggering conditions. This mechanism makes the execution flow of tasks no longer dependent on a preset fixed order, but dynamically determined by the semantic relationships between SMUs and their current satisfaction status. As a result, the system can flexibly handle tasks with complex dependencies and conditional branches, and can automatically and efficiently re-evaluate the executable status of the entire task flow when some task modules fail to execute, external conditions change, or the task structure is updated online, realizing true condition-driven and event-driven dynamic scheduling. Thirdly, the Semantic-Driven Scheduling Execution Engine (SCDE) uniformly schedules and distributes SMUs that meet the conditions, and triggers the recalculation and state propagation of the semantic satisfaction conditions in the task structure after each SMU is executed. This forms a closed-loop execution-feedback-rescheduling control loop. This loop ensures that changes in the task execution status (e.g., completion, failure) can affect the executability evaluation of other related SMUs in real time and automatically according to the definition of SR, thereby making the task execution process of the entire system highly responsive and coherent. In summary, the technical solution of this application can effectively solve the problems of rigid task representation, difficulty in integration with AI systems, and insufficient dynamic adaptability in the prior art, and significantly improve the intelligence level and robustness of embodied intelligent systems in performing complex and dynamic motion tasks.

[0118] Figure 3 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. For example... Figure 3As shown, the electronic device 3 in this embodiment mainly includes: a processor 30, a memory 31, and a computer program 32 stored in the memory 31 and executable on the processor 30, such as a program for a semantic execution method for motion tasks. When the processor 30 executes the computer program 32, it implements the steps in the above-described embodiment of the semantic execution method for motion tasks, for example... Figure 1 Steps S1 to S4 are shown. Alternatively, when processor 30 executes computer program 32, it implements the functions of each module / unit in the above-described device embodiments, for example... Figure 2 The functions of the acquisition module 201, mapping module 202, execution module 203, and update module 204 are shown.

[0119] For example, the computer program 32 of the semantic execution method for motion tasks mainly includes: obtaining the task structure, wherein the task structure includes a semantic representation unit (SMU) and a semantic relation (SR), wherein the SMU is used to represent a minimal schedulable execution semantic unit of motion operations, and the SR is used to define the semantic association between at least two SMUs in terms of execution timing or constraints; receiving planning intent information, and mapping the planning intent information to new SMUs and new SRs conforming to the task structure format through the AI ​​planning intent mapping (AIM) interface to update the task structure, wherein the planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences; and driving the process based on semantic satisfaction conditions (SSC). The scheduling process involves repeatedly executing steps S31 and S32 until all SMUs are executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the pending execution state. For the current SMU, check whether all its incoming SRs satisfy the triggering conditions. If all are satisfied, determine that the semantic satisfaction condition SSC of the SMU is met, and update its state to ready. Step S32: Submit the SMUs in the ready state to the semantically driven scheduling execution engine SCDE for scheduling and distribution. Execute the motion operations defined by the SMUs distributed by SCDE, and update the state of the SMU after execution to trigger the recalculation of the semantic satisfaction conditions and state propagation in the task structure. The computer program 32 can be divided into one or more modules / units, which are stored in the memory 31 and executed by the processor 30 to complete this application. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 32 in the electronic device 3.For example, computer program 32 can be divided into the functions of acquisition module 201, mapping module 202, execution module 203, and update module 204 (a module in the virtual device). The specific functions of each module are as follows: Acquisition module 201 is used to acquire the task structure, wherein the task structure includes semantic representation unit (SMU) and semantic relation (SR). The SMU is used to represent a minimal schedulable execution semantic unit of motion operation, and the SR is used to define the semantic association between at least two SMUs in terms of execution timing or constraints; Mapping module 202 is used to receive planning intent information and map the planning intent information into new SMUs and new SRs conforming to the task structure format through the AI ​​planning intent mapping (AIM) interface to update the task structure, wherein the planning intent information includes natural language instructions, structured data, and motion trajectory sequences. Any one or more; Execution module 203 is used to drive scheduling based on semantic satisfaction condition SSC, and cyclically execute the following steps S31 and S32 until all SMUs are executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the state of pending execution. For the current SMU, check whether all its incoming SRs meet the triggering conditions. If they all meet the conditions, it is determined that the semantic satisfaction condition SSC of the SMU is established, and its state is updated to ready; Step S32: Submit the SMUs in the state of ready to the semantic-driven scheduling execution engine SCDE for scheduling and distribution; Update module 204 is used to execute the motion operation defined by the SMU distributed by SCDE, and update the state of the SMU after execution to trigger the recalculation of the semantic satisfaction condition and state propagation in the task structure.

[0120] Electronic device 3 may include, but is not limited to, processor 30 and memory 31. Those skilled in the art will understand that... Figure 3 This is merely an example of electronic device 3 and does not constitute a limitation on electronic device 3. It may include more or fewer components than shown, or combine certain components, or different components. For example, electronic device may also include input / output devices, network access devices, buses, etc.

[0121] The processor 30 may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0122] The memory 31 can be an internal storage unit of the electronic device 3, such as a hard disk or RAM. The memory 31 can also be an external storage device of the electronic device 3, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card. Furthermore, the memory 31 can include both internal and external storage units of the electronic device 3. The memory 31 is used to store computer programs and other programs and data required by the electronic device. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0123] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed. That is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above-described device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0124] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0125] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0126] In the embodiments provided in this application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0127] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0128] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0129] If integrated modules / units are implemented as software functional units and sold or used as independent products, they can be stored in a storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments can also be implemented by a computer program instructing related hardware. The computer program for the semantic execution method of motion tasks can be stored in a storage medium. When the computer program is executed by a processor, it can implement the steps of the above method embodiments, namely, obtaining the task structure, wherein the task structure includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU is used to represent a minimal schedulable motion operation semantic unit, and the SR is used to define the semantic association between at least two SMUs in terms of execution timing or constraints; receiving planning intent information, and mapping the planning intent information to new SMUs and new SRs conforming to the task structure format through the AI ​​planning intent mapping (AIM) interface to update the task structure, wherein the planning intent information includes The process involves one or more of the following: natural language instructions, structured data, and motion trajectory sequences. Based on semantic satisfaction condition (SSC) driven scheduling, the following steps S31 and S32 are executed iteratively until all SMUs are executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the pending execution state. For the current SMU, check whether all its incoming SRs satisfy the triggering conditions. If all are satisfied, the semantic satisfaction condition (SSC) of the SMU is determined to be true, and its state is updated to ready. Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling execution engine (SCDE) for scheduling and distribution. Execute the motion operations defined by the SMUs distributed by SCDE, and update the state of the SMU after execution to trigger the recalculation of the semantic satisfaction conditions and state propagation in the task structure. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or some intermediate form. Storage media can include: any entity or device capable of carrying computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the contents of storage media can be appropriately added to or removed according to the requirements of legislation and patent practice in a jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, storage media may not include electrical carrier signals and telecommunication signals.

[0130] The above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. These modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application. The specific embodiments described above further illustrate the purpose, technical solutions, and beneficial effects of this application. It should be understood that the above descriptions are merely specific embodiments of this application and are not intended to limit the protection scope of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

Claims

1. A semantic execution method for motion tasks, characterized in that, The method includes: Step S1: Obtain the task structure, which includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU represents a minimum schedulable execution semantic unit of motion operations, and the SR defines the semantic association between at least two SMUs in terms of execution timing or constraints. Step S2: Receive planning intent information, and map the planning intent information into new SMU and new SR that conform to the task structure format through the AI ​​planning intent mapping AIM interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences. Step S3: Based on the semantic satisfaction condition, SSC-driven scheduling is executed repeatedly in steps S31 and S32 until all SMUs have been executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the state of pending execution. For the current SMU, check whether all its incoming SRs meet the triggering conditions. If they all meet the conditions, determine that the semantics of the SMU meet the SSC condition and update its state to ready. Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling and execution engine SCDE for scheduling and distribution; Step S4: Execute the motion operation defined by the SMU distributed by the SCDE, and update the state of the SMU after execution to trigger the recalculation and state propagation of the semantic satisfaction conditions in the task structure.

2. The semantic execution method for motion tasks according to claim 1, characterized in that, The SMU includes at least the unit_type and action_spec fields, and the SR includes at least the relation_type field. The process of mapping the planning intent information into new SMUs and new SRs conforming to the task structure format via the AI ​​Planning Intent Mapping (AIM) interface includes: Step S21: Analyze the planning intent information to identify the motion elements, logical relationships between actions, and constraints. Step S22: Based on the preset mapping rule library, instantiate the identified motion action elements into SMUs with specific unit_type and action_spec; Step S23: Based on the identified logical relationships and constraints, generate an SR that connects to the corresponding SMU. The relation_type of the SR includes at least one of sequential dependency (SEQ), parallel start (PAR), synchronization barrier (SYNC), and condition triggering (COND).

3. The semantic execution method for motion tasks according to claim 2, characterized in that, The SR includes at least the relation_type and condition fields. When the relation_type of the generated SR is condition-triggered COND, the method further includes: Step S231: Configure trigger conditions based on sensor signals, system state variables, or logical expressions in the condition field of the SR; Step S232: During the scheduling process, the triggering condition is continuously monitored, and the SR of the COND type is determined to meet the triggering condition only when the triggering condition is met.

4. The semantic execution method for motion tasks according to claim 1, characterized in that, The SR includes at least the relation_type and source_units fields, and the triggering condition is dynamically determined based on the relation_type of the SR in the following way: Step S311: For an SR with relation_type of sequential dependency SEQ, the triggering condition is that the execution status of all SMUs in the source_units of the SR is completed; Step S312: For an SR with relation_type set to SYNC (synchronization barrier), the trigger condition is that the execution status of all SMUs in the source_units of the SR is completed. Step S313: For an SR with relation_type as resource mutual exclusion RES_MUTEX, the triggering condition is that all SMUs in the source_units of the SR have released their occupation of the specified resource.

5. The semantic execution method for motion tasks according to claim 1, characterized in that, The method further includes performing the following dynamic updates during the execution of step S3: Step S5: Receive dynamic update operation DUO instructions for the runtime task structure, wherein the DUO instructions include at least one of inserting, deleting, replacing SMU or SR; Step S6: Before executing the DUO instruction, verify its consistency with the current task structure and execution state to ensure that the execution semantics of the completed SMU are not violated; Step S7: After passing the consistency verification, apply the DUO instruction to modify the task structure and immediately trigger the recalculation of the semantic satisfaction condition SSC of the affected SMU.

6. The semantic execution method for motion tasks according to claim 1, characterized in that, The SMU includes at least a recovery_path field, and the method further includes the following process for executing the error recovery path: Step S8: When it is detected that the execution status of an SMU has been updated to failure, query the recovery strategy defined in the recovery_path field of that SMU; Step S9: Execute the corresponding recovery operation according to the recovery strategy. The recovery operation includes: retrying the current SMU, skipping the current SMU and scheduling its subsequent SMU, reverting to the previous SMU specified in the task structure to start again, or triggering an exception handling subtask process.

7. The semantic execution method for motion tasks according to claim 6, characterized in that, The recovery strategy is to trigger an exception handling subtask process, and step S9 includes: Step S91: Load the predefined exception handling subtask template based on the identifier in the recovery_path field; Step S92: Based on the context information of the currently failed SMU, parameterize and instantiate the exception handling subtask template to generate a subtask structure containing several SMUs and SRs. Step S93: Insert the subtask structure into the current task structure and establish its semantic connection with the original task structure. Then, the SCDE schedules and executes the subtask structure.

8. A semantic execution device for motion tasks, characterized in that, The device includes: The acquisition module is used to acquire the task structure, which includes a semantic representation unit (SMU) and a semantic relationship (SR). The SMU is used to represent a minimum schedulable execution semantic unit of motion operation, and the SR is used to define the semantic association between at least two SMUs in terms of execution timing or constraints. The mapping module is used to receive planning intent information and map the planning intent information into new SMU and new SR that conform to the task structure format through the AI ​​Planning Intent Mapping (AIM) interface to update the task structure. The planning intent information includes any one or more of natural language instructions, structured data, and motion trajectory sequences. The execution module is used to drive scheduling based on semantic satisfaction condition SSC, and repeatedly execute the following steps S31 and S32 until all SMUs are executed or the task is terminated: Step S31: Traverse the SMUs in the task structure that are in the pending state. For the current SMU, check whether all its incoming SRs meet the triggering condition. If they all meet the condition, it is determined that the semantic satisfaction condition SSC of the SMU is met, and its state is updated to ready. Step S32: Submit the SMUs in the ready state to the semantic-driven scheduling execution engine SCDE for scheduling and distribution. The update module is used to execute the motion operation defined by the SMU distributed by the SCDE, and update the state of the SMU after execution to trigger the recalculation and state propagation of the semantic satisfaction conditions in the task structure.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as described in any one of claims 1 to 7.

10. A storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method as described in any one of claims 1 to 7.

Images